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FOREWORD 


The  Thirty-Sixth  Conference  of  the  Design  of  Experiments  in  Army 
Research,  Development  and  Testing  was  held  17-19  October  1990  on 
the  campus  of  the  University  of  Delaware.  This  university  served 
as  one  of  its  hosts,  the  other  host  being  the  Ballistic  Research 
Laboratory  (BRL) .  Back  in  1987  the  Thirty-Third  Design  Conference 
had  these  same  hosts.  Professor  Henry  B.  Tingey  was  the 
Chairperson  on  Local  Arrangements  for  the  University  and  Mr.  Jerry 
Thomas  served  in  this  capacity  for  BRL.  The  members  of  the  Army 
Mathematical  Steering  Committee  (AMSC) ,  sponsor  of  these 
conferences,  take  this  opportunity  to  thank  these  gentlemen  for 
their  excellent  handling  of  the  many  problems  associated  with  this 
meeting. 

The  original  format  for  the  Design  of  Experiments  Conferences, 
which  are  under  the  auspices  of  the  AMSC,  was  outlined  by  the 
eminent  statistician,  Professor  Samuel  S.  Wilks,  who  served  as 
conference  chairman  until  his  death.  Through  these  symposia  the 
AMSC  hopes  to  introduce  and  encourage  the  use  of  the  latest 
statistical  and  design  techniques  into  the  research,  development 
and  testing  conducted  by  the  Army's  scientific  and  engineering 
personnel . 

Members  of  the  program  committee  were  pleased  to  obtain  the 
services  of  the  following  distinguished  scientists  to  speak  on 
topics  of  interest  to  Army  personnel. 


KEYNOTE  ADDRESS 

Design  of  Experiments  for  comparing  the  Performance  of  Several 
Multi-Stage  Procedures  for  Selecting  the  Normal  Population  Having 
the  Largest  Mean  When  the  Populations  Have  a  Common  Variance. 

Professor  Robert  E.  Bechhofer  Professor  David  M  Goldsman 

Cornell  University  Georgia  Institute  of  Technology 
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Dr.  Paul  A.  Turkey 
Bell  Communication 
Research 

Professor  Howard  M.  Taylor 
University  of  Delaware 

Professor  Erik  V.  Nordheim 
University  of  Wisconsin 

Professor  Arthur  E.  Hoerl 
University  of  Delaware 

Each  year,  two  days  before  the  start  of  the  conference,  a  two-day 
tutorial  is  presented.  This  year  Professor  Russell  R.  Barton,  now 
at  The  Pennsylvania  state  University,  presented  a  tutorial  entitled 
"Graphical  Design  of  Experiments."  It  was  held  on  the  campus  of 
the  University  of  Delaware.  His  notes  on  this  meeting  are  being 
published  in  these  proceedings. 

Since  no  U.S.  Army  Wilks  Award  would  be  given  in  1990,  the  hosts 
decided  to  have  an  invited  speaker  after  the  banquet.  Professor 
Tingey  invited  Dr.  John  C.  Bailar,  Department  of  Health  and  Human 
Services,  to  talk  on  "John  C.  Bailar' s  Laws  of  Data  Analysis."  It 
turned  out  to  be  one  of  those  rare  talks  that  all  members  in  the 
audience  could  enjoy. 

The  AMSC  has  requested  that  these  transactions  be  published  and 
distributed  Army-wide  so  that  the  information  in  them  might  assist 
Army  scientists  with  some  of  their  statistical  problems.  Committee 
members  would  like  to  thank  all  the  speakers  for  their  interesting 
presentations  and  also  members  of  the  Program  Committee  for  their 
many  contributions  to  this  scientific  meeting. 
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DESIGN  OF  EXPERIMENTS  FOR  COMPARING  THE 
PERFORMANCES  OF  SEVERAL  MULTI-STAGE  PROCEDURES 
FOR  SELECTING  TIIE  NORMAL  POPULATION  HAVING  THE 
LARGEST  MEAN  WHEN  TIIE  POPULATIONS  HAVE  A 
COMMON  VARIANCE* 


Robert  E.  Bechhofer 
School  of  Operations  Research  k 
Industrial  Engineering 
College  of  Engineering,  Cornell  University 
Ithaca,  NY  14853 


David  M.  GoldsniAn 

School  of  Industrial  k  Systems  Engineering 
Georgia  Institute  of  Technology 
Atlanta,  GA  30332 


Abatrash 


We  study  the  performance  characteristics  of  indifference-zone  procedures  for  selecting  the  normal 
population  which  has  the  largest  mean  when  the  populations  have  a  common  known  variance.  The 
procedures  considered  arc  the  open  sequential  procedure  of  Bechhofer,  Kiefer  and  Sobel  and  a  truncated 
version  of  that  procedure  by  Bechhofer  and  Goldsman,  the  closed  multi-stage  procedure  of  Paulson 
which  eliminates  populations  indicated  as  being  non-contending,  and  an  improved  version  of  that 
procedure  by  Hartmann.  The  performance  characteristics  studied  are  the  achieved  probability  of  a 
correct  selection,  the  expected  number  of  stages  required  to  terminate  experimentation,  and  the 
expected  total  numbor  of  observations  required  to  terminate  experimentation.  All  performance 
characteristics  are  estimated  by  Monte  Carlo  sampling.  In  addition,  the  same  problem  is  considered  for 
the  case  of  common  unknown  variance.  Here  the  competing  procedures  are  the  open  non-eliminating 
two-stago  procedure  of  Bechhofer,  Dunnett  and  Sobel,  the  open  eliminating  two-stage  procedure  of 
Gupta  and  Kim,  and  the  open  eliminating  multi-stage  sequential  procedure  of  Paulson  as  improved  by 
Hartmann.  Particular  emphasis  is  placed  on  the  fact  that  when  designing  the  experiments  to  compare 
the  performance  characteristics  of  these  procedures  there  are  many  relevant  factors  which  must  be 
varied  in  the  conduct  of  the  experiment  in  order  that  meaningful  and  generalizable  results  can  be 
obtained, 


Rescarcli  partially  supported  by  the  U.S.  Army  ilesearcli  Office  through  the  Mathematical  Sciences 
Institute  of  Cornell  University, 


Bechhofer  ( L954)  proposed  a  statistical  procedure  for  selecting  the  normal  population  which  has 
the  largest  population  mean  with  prespecified  controls  over  the  probability  of  a  correct  selection.  Over 
the  years  many  additional  procedures  have  been  proposed  to  solve  this  problem.  All  of  these 
procedures  guaranteed  the  following  indifference-zone  probability  requirement: 


Prob  {Correct  selection}  >  P*  whenever  -  /j^  >  6* 


Here  «C  £  •••  S:  are  the  values  of  the  population  means  (1  <  i  <,  k);  the 

values  of  the  pj  and  (1  <  i,  j  <  k)  are  assumed  to  be  unknown  as  is  the  pairing  of  the  /iyj 
(1  <  k)  with  the  k  populations.  The  constants  {£,  P*}  with  0  <  6*  <  oo,  1/k  <  P*  <  1  are 
specified  by  the  experimenter  prior  to  the  start  of  experimentation. 

Each  of  these  procedure  at  the  time  that  it  was  suggested  possessed  certain  virtues  which  made  it 
worthy  of  consideration.  But  it  was  not  until  recent  years  that  any  serious  attempt  was  made  to 
compare  the  relative  merits  of  these  procedures.  The  ultimate  objectives  of  such  a  study  would  be  to 
determine  whether  any  particular  procedure  dominated  any  other  procedure(s),  and  if  so,  in  what 
respect  and  under  what  conditions.  Such  comparative  studies  could  then  make  it  possible  to 
recommend  which  procedure  it  would  be  preferable  to  use  in  certain  particular  environments. 

The  first  such  comprehensive  study  was  reported  by  Bechhofer  and  Goldsman  (1989);  in  that 
study  which  dealt  with  the  cases  in  which  the  populations  have  a  common  known  variance,  the  critical 
performance  characteristics  were  compared  for  the  single-stage  procedure  of  Bechhofer  (1954),  the  two- 
stage  procedure  of  Tamhane  and  Bechhofer  (1977,  1979)  in  which  populations  indicated  as  being  non¬ 
contending  can  be  eliminated  after  the  first  stage,  the  open  non-eliminating  sequential  procedure  of 
Bechhofer,  Kiefer  and  Sobel  (1068),  and  the  truncated  version  of  that  procedure  by  Bechhofer  and 

i 

Goldsman  (1987,  1989),  and  the  closed  multi-stage  procedure  of  Paulson  (1964)  in  which  population 
can  be  eliminated  at  any  stage  after  the  first,  and  an  improved  version  of  that  procedure  by  Hartmann 
(1988).  Most  recently  Bechhofer,  Dunnett,  Goldsman  and  Hartmann  (1990)  studied  the  same  selection 
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problem  for  the  case  of  common  unknown  variance.  The  procedures  considered  were  the  open  non¬ 
eliminating  two-stage  procedure  of  Bechhofer,  Dunnett  and  Sobel  (1954),  the  open  two-stage  procedure 
of  Gupta  and  Kim  (1984)  in  which  non-contending  population  can  be  eliminated  after  the  first  stage, 
and  the  open  multi-stage  procedure  of  Paulson  (1964)  as  improved  by  Hartmann  (1991)  in  which 
population  can  be  eliminated  at  every  stage  after  the  first. 

In  the  talk  presented,  results  and  conclusions  for  the  case  of  common  known  variance  were 
discussed  in  great  detail.  It  was  pointed  out  why  the  case  of  common  unknown  variance  introduced 
special  design  problems  in  terms  of  the  large  factorial  experiment  to  be  conducted.  The  final  factorial 
experiment  as  it  was  conducted  is  described  below.  The  results  of  the  experiment  are  reported  in 
Bechhofer,  Dunnett,  Goldsman  and  Hartmann  (1900). 


Design  of  a  Large  Factorial  Experiment  to  Compare  the 
Relative  Performances  of  Four  Multi-Stage  Procedures  for  Selecting  the  Population  with  the  Largest 

Mean  when  the  Common  Variance  is  Unknown 


The  procedures! 

a)  Bechhofer-Dunnett-Sobel  (1054) 

b)  Paulson-Hartmann  with  A  *  5*/4  (1991) 

c)  Paulson-Hartmann  with  A  =  6*/ 2  (1991) 

d)  Gupta-Klm  (1984) 

New  underlying  factor: 

The  value  of  the  unknown  variance  which  plays  its  role  in  terms  of  the  ratio  a/ 6*  where  6*  >  0 
is  specified. 

New  design  factor; 

This  is  under  the  control  of  the  experimenter.  Common  number  of  observations  (n^)  per 
population  which  is  taken  in  the  first  stage. 

Factors  affecting  the  performances  each  procedure! 

a)  Number  of  populations:  These  were  set  at  three  levels: 
k=3,5,10. 

b)  Specified  P*-values:  These  were  set  at  2  levels: 

P*s  0.75,  0.90. 

c)  Ratios  c/6*\  These  were  set  at  laa  levels:  er/6*=s2,3. 

d)  First-stage  sample  sizes:  These  were  set  at  levels: 

-s  5,  10,  15,  20. 

e)  Configuration  of  the  population  means:  These  were  set  at 
£aa  levels:  least-favorable  and  equal  means. 

Thus  for  each  procedure  a  5-factor  experiment  was  conducted  with  3x2x2x4x2  =  06  factor-level 
combinations.  For  each  combination  at  least  10,000  independent  MC  experiments  were  conducted. 

The  re? ponies  recorded  for  each  experiment  were: 

a)  The  achieved  probability  of  a  correct  selection. 

b)  The  estimated  expected  total  number  of  observations  to 
terminate  sampling. 

c)  The  estimated  variance  of  the  total  number  of 
observations  to  terminate  sampling. 

d)  The  estimated  expected  number  of  vector-observations  to 
terminate  sampling. 

e)  The  estimated  variance  of  the  number  of  vector- 
observations  to  terminate  sampling. 
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ARTILLERY  COMPUTER  METEOROLOGICAL  MESSAGE 
ZONE  THICKNESS  FOR  ALTITUDES  ABOVE  20  KM 


Abel  J .  Blanco 

U.S.  Army  Atmospheric  Sciences  Laboratory 
White  Sands  Missile  Range,  NM  88002-5501 


ABSTRACT .  The  U,S,  Army  is  developing  new  field  artillery  systems  that 
have  longer  range  (30  km)  capabilities.  As  a  result,  extended  altitude 
computer  meteorological  (met)  messages  are  now  required  for  aiming 
adjustments  that  compensate  for  met  effects  along  the  high  apogee 
trajectory.  The  following  experiment  is  designed  to  select  an  optimal  zone 
thickness:  aerodynamic  parameters  for  an  artillery  rocket  assisted  round 
ate  assumed.  and  the  Ballistic  Research  Laboratory  general  trajectory  model 
is  used  to  simulate  impacts  computed  from  using  observed  met  data  and 
observed  data  averaged  into  proposed  zone  thicknesses  for  the  altitudes 
extending  above  the  current  20  km  computer  met  message  height.  Measured 
upper-air  wind,  temperature,  and  pressure  data  collected  during  the  four 
seasons  at  White  Sands  Missile  Range,  New  Mexico,  are  converted  for  use  in 
artillery  surface-to-surface  applications.  The  226  rawinsonde  flights, 
including  data  up  to  30  km  height,  are  partitioned  into  monthly  data  sets, 
and  tests  of  significance  are  used  to  determine  an  optimal  thickness.  The 
corresponding  impact  paired  differences  between  the  observed  data  and  the 
proposed  zone  averaged  data  yield  statists  ..  that  demonstrate  the  1-km  zone 
average  as  the  optimally  selected  thickness  for  extending  the  computer  met 
message. 


1.  INTRODUCTION.  The  U.S.  Army  Atmospheric  Sciences  Laboratory  (AS1.) 
continues  to  support  the  U.S.  Army  Field  Artillery  School  (USAFAS)  in 
extending  the  application  of  the  artillery  computer  meteorological  (met) 
message  at  altitudes  above  20  km.  In  lieu  of  unavailable  measured  data,  a 
software  technique  (Blanco,  1981)  was  developed  to  extend  the  application  to 
3  km  from  the  maximum  ordinate.  By  1983,  ASL  refined  a  composite  algorithm 
allowing  use  of  estimated  higher  altitude  met  data.  Persistence  of  the  last 
computer  met  message  line  data  and  satellite  climatology  or  available 
artillery  fallout  met  message  were  merged  into  an  automated  procedure  that 
selects  the  best  options  for  extending  the  artillery  computer  met  message. 
The  USAFAS  now  has  a  new  requirement  of  extending  the  computer  met  message 
to  a  maximum  ordinate  of  30  km.  The  10  km  extension  dictates  using  measured 
data  Instead  of  the  early  proposals  of  using  estimated  data.  The  only 
constraint  is  that  one  can  easily  overload  the  fire  control  computer  with 
too  much  met  data.  The  current  met  array  allows  27  observations  for  each 
met  parameter --wind  direction,  windspeed,  virtual  temperature,  and  pressure. 
These  27  lines  remain  the  same  and  the  new  fire  control  computer  storage 
requirements  are  determined  upon  selecting  the  optimal  zone  thickness  that 
best  describes  the  atmospheric  state  between  20  km  and  30  km  above  the 
surface.  The  only  expected  change  in  the  field  operations  is  to  continue 
collecting  met  data  up  to  30  km  above  the  surface.  All  other  required 
software  changes  will  be  transparent  to  the  user. 

This  paper  presents  an  optimal  zone  thickness  for  the  extended 
artillery  computer  met  message  at  altitudes  above  20  km.  The  method  used  in 
selecting  the  zone  thickness  that  can  be  standardized  for  artillery 
applications  includes  the  following  experiment:  aerodynamic  parameters  for 
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an  artillary  rockat  aaalatad  round  are  assumed  and  the  Ballistic  Research 
Laboratory  (BRL)  general  trajectory  model  (Lieske  and  Reiter,  1966)  is  used 
to  simulate  impacts  computed  from  using  observed  met  data  and  observed  data 
averaged  into  proposed  zone  thicknesses  for  the  altitudes  extending  above 
the  current  20  km  computer  met  message  height.  The  corresponding  impact 
paired  statistics  between  the  observed  and  proposed  zone  thicknesses  are 
used  to  document  the  selection  of  the  optimal  zone  thickness. 

The  analysis  of  these  results  includes  mean  component  Inferences.  The 
Student  t-test  and  the  F-ratio  analysis  of  variance  comparison  are  used  to 
determine  a  significant  difference  between  the  observed  and  the  1-km  zone 
thickness  and  between  the  observed  and  the  2-km  zone  thickness.  A  mean 

o 

vector  Inference  test  known  as  the  Hotelling's  T  is  also  used  for  an  easier 
Interpretation  of  results.  This  test  also  allows  faster  calculations  when 
using  available  utilities  from  the  International  Mathematics  and  Science 
Library  (IMSL) .  In  summary  these  statistics  demonstrate  the  1-km  zone 
average  as  the  optimally  selected  thickness  for  extending  the  computer  met 
message.  With  this  zone  thickness,  10  new  lines  are  added  to  the  computer 
met  message. 

2.  DESIGN  OF  EXPERIMENT .  The  U.S.  Army  Field  Artillery  is  developing 
new  weapon  systems  that  have  a  longer  range  (assume  50  km)  capability.  To 
reach  this  new  range  the  projectile  must  go  through  a  higher  apogee.  The 
projectile  propagates  through  more  atmosphere  and  spends  a  longer  time  under 
the  influence  of  atmospheric  parameters  of  wind,  temperature,  and  pressure. 
Since  met  effects  are  known  to  be  major  contributors  to  the  artillery  error 
budget,  deployment  of  met  teams  in  the  battle  area  is  required  to  measure 
the  atmospheric  parameters,  prepare  formatted  messages,  and  disseminate 
computer  met  messages  to  the  fire  control  centers.  These  messages  are  used 
in  making  final  aiming  angle  adjustments  before  ordering  fire  for  effect. 

Figure  1  presents  a  two-dimensional  plot  of  height  versus  range  for  the 
two  modes  of  fire --low  quadrant  angle  and  high  quadrant  angle.  New 
aerodynamic  and  ballistic  coefficients  at  the  higher  mach  numbers  were 
Incorporated  in  the  BRL  general  trajectory  model  to  simulate  firings  of  a 
modified  M549A1  rocket  assisted  round.  The  assumed  future  configuration  of 
a  155 -mm  weapon  system  fired  at  White  Sands  Missile  Range,  New  Mexico,  did 
not  reach  an  apogee  of  30  km.  For  the  low-angle  simulations,  the  projectile 
apogee  is  at  23  km  above  the  surface.  For  the  high-angle  simulations,  the 
projectile  apogee  is  at  26  km,  Of  course  the  simulation  can  perform  the  30 
km  apogee;  but  at  this  higher  angle  of  fire  the  BRL  test  on  maximum  angle  of 
attack  exceeds  a  predefined  criterion,  which  Indicates  that  the  projectile 
1b  not  trailing  the  trajectory  and  there  is  the  possibility  of  actual  round 
tumble.  Following  the  BRL  criteria,  this  report  evaluates  results  from 
simulations  containing  23  to  26  km  apogees  corresponding  to  the  low  end  high 
quadrant  angles. 


Robert  Lieske,  1990,  Memorandum  SGLBR-LF-T  (340),  "Maximum  Ordinates  for 
Extended  Range  Ordnance,"  U.S.  Army  Ballistic  Research  Laboratory,  Aberdeen 
Proving  Ground,  Maryland. 
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Figure  1.  Assumed  155 -nun  cannon  projectile  trajectory. 

All  effective  fire  includea  met  adjuatmenta,  and  table  1  preaents  the 
zone  thickneaaes  for  three  artillery  met  meaaagea.  Today' a  artillery 
meteorology  still  dependa  on  a  balloon-borne  aenaor  to  provide  the 
appropriate  wind,  temperature,  and  pressure  measurements.  In  computing  the 
fallout  message  for  example,  the  balloon  is  tracked  up  to  30  km  above  the 
surface.  The  current  computer  met  message  provides  met  data  up  to  only  20 
km  because  cannon  artillery  has  not  exceeded  this  maximum  ordinate.  Special 
artillery  rockets  have  the  capability  of  exceeding  this  altitude,  and  the 
USAFAS  has  requested  methods  of  extending  the  application  of  the  available 
computer  met  message.  To  thi"’  date  the  software  estimations  have  been 
appropriate  only  because  the  required  extension  was  within  4  km  of  the  last 
observed  computer  line  data.  With  the  advent  of  future  artillery  systems, 
there  is  a  need  to  extend  the  computer  met  message  to  a  maximum  ordinate  of 
30  km.  Met  hardware  and  software  are  already  available  and  the  only  task  is 
to  standardize  the  zone  thickness  within  the  20  to  30  km  altitudes, 

This  task  has  been  a  cooperative  effort  between  the  UASFAS,  BRL,  and 
ASL.  ASL  recommended  all  1-min  observations;  however,  the  USAFAS  is 
constrained  to  available  computer  storage  space  in  the  fire  control  centers 
and  cannot  support  this  -300  m  thickness.  BRL  has  assumed  aerodynamic 
characteristics  and  provided  ASL  with  the  point  mass  trajectory  model  for 
use  in  computing  the  desired  long-range  cannon  fire.  An  experiment  is  then 
designed  to  prepare  met  messages  that  contain  1-min  observed  data,  1-km  zone 
averages,  and  2-km  zone  averages,  These  formatted  data  are  then  input  to 
the  BRL  point  masB  general  trajectory  model  for  simulating  impacts.  All 
replicates  are  then  pa'red  and  statistical  results  are  interpreted  to 
determine  if  proposed  averaged  data  and  the  observed  data  are  significantly 
different . 
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TABLE  1.  FM  6-15  ATMOSPHERIC  ZONE  STRUCTURE 
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3.  METEOROLOGICAL  DATA  BASE.  An  available  «et  of  upper-air  data 
containing  observations  up  to  30  km  was  retrieved  from  met  support  provided 
for  special  projects  at  White  Sands  Missile  Range,  New  Mexico,  during  1989. 
A  sample  of  7.26  rawinsonde  flights  containing  representative  sets  for  each 
of  the  four  seasons  is  used  as  the  met  data  base.  Table  2  presents  all 
replicate  rawinsonde  flights  with  Holloman,  White  Sands,  and  Small  Missile 
Range  containing  the  larger  samples. 


10 


TABLE  2,  NUMBER  OF  RAW1NSONDE  FLIGHTS  COLLECTED  DURING  1989 


MONTHLY  FLIGHTS 
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Each  rawinsonde  flight  in  thli  data  bait  was  raduoad  Into  tha  26 
computar  zona  avaragaa  by  ualng  a  linaar  fit  mathodology .  Glvan  tha 
particular  data  (W  aquala  wind,  tanparatura,  or  praaaura)  varaua  height,  ona 
can  Interpolate  for  any  information  within  the  adjacent  observed  data  by 
using  tha  linaar  coefficients  (a  and  b) i 


_  b  _  mi  zd  t  n  -  m  t  n  my 

Z(1  +  1)  -  Z(i)  '  D  Z(i  +  1)  -  Z(i) 


u> 


where  tha  range  of  application  is  within  Z(i>  and  Z(i  +  1).  Then  for  a 
given  height  H  between  Z(i)  and  Z(1  +  1),  one  con  compute  the  value  of  the 
met  parameter  W  by 

W  -  a  H  +  b.  (2) 


With  this  linear  coefficient  method,  one  can  derive  the  value  of  the  mot 
parameter  at  the  bottom  and  top  of  any  defined  zona  thickness.  Figure  2 
illustrates  the  position  of  the  data  (o)  and  the  location  of  the  artillery 
zones  (H(i)'s).  The  total  AREA  defined  by  the  vertical  axis,  the  bottom 
horizontal  line  at  height  H(l),  the  top  horizontal  line  at  height  H(2),  and 
the  met  parameter  function  within  the  interval  H(l)  and  H(2)  is  calculated 
by  the  sum  of  the  composite  areas.  For  example, 


dwdh 


■I 


(ah  +  b)  dh 


-  f +  bh 


Z<2) 

HU) 


(3) 


computes  the  area  by  integration  between  H(l)  and  Z(2).  Repeating  the 
interation  between  Z(2)  and  Z(3)  and  Z(3)  and  H(2)  yields  the  composite 
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areas ,  and  the  aunt  of  thaaa  areas  defines  the  total  AREA.  Ualng  the  Mean 
Value  Theorem,  one  can  derive  the  mean  value  of  the  particular  parameter 
within  the  artillery  zone  thickness  by 


n  AREA 

w  AH 


(4) 


Since  the  atmoapheric  preaaure  la  exponentially  related  to  the  height,  the 
natural  log  of  the  preaaure  ia  computed  before  the  linear  fit  and  averaging 
routines  are  executed.  At  the  end,  the  exponential  function  ia  applied  to 
the  output  to  express  the  zone  pressure  avorage. 


(wind,  temptraturt,  praasura) 


Figure  2.  Met  data  and  zone  thickness. 

In  computing  the  wind  zono  averages,  the  artillery  interpolates  for  the 
x  and  y  balloon  positions  at  the  bottom  and  the  top  of  each  artillery  zone 
then  divides  by  the  time  the  balloon  ascends  from  the  bottom  to  the  top  of 
the  zone.  Note  from  table  1  that  the  computer  zones  are  not  the  same 
thickness.  Since  actual  computer  met  messages  are  not  available,  the  best 
estimate  for  reproducing  the  artillery  met  message  from  the  available 
rawinsonde  data  is  that  described  in  this  section. 

Figures  3  through  5  present  reduced  computer  met  messages  extended  to  a 
higher  altitude  with  the  actual  observed  data  and  the  two  proposed  zone 
averages .  Note  that  line  0  or  the  surface  is  at  the  Holloman  site  elevation 
of  1258  m  and  the  top  of  line  26  is  at  21238  m.  In  the  experiment  this 
structure  is  fixed  and  the  only  variable  is  the  data  from  the  top  of  line  26 
to  the  last  available  met  observation.  These  plots  present  the  observed 
wind  component  profiles  for  the  December  22  flight  at  the  extended  computer 
met  message  heights.  The  observed  data  represents  1  min  or  about  300  m 
interval  observations.  Using  the  information  in  figure  3  as  input  to  the 
BRL  general  trajectory  model,  one  can  compute  a  43506-m  range  and  a  5137 -m 
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cross* component  impact  for  the  assumed  future  projectile  fired  at  a  high 
quadrant  angle  of  1150  mils,  In  figure  4,  the  extended  observed  data  is 
averaged  into  1  km  zones,  and  the  simulated  Impacts  demonstrate  almost 
identical  results,  However,  when  one  averages  the  observed  data  into  2  km 
zones,  as  represented  in  figure  5,  the  simulated  impacts  demonstrate  a  -64  m 
range  and  453  m  cross  difference, 
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Figure  3. 


WIND  COMPONENTS  (m/s) 

Computer  wind  components  extended  with  observed  data, 


Figure  4.  Computer  wind  components  extended  with  1  km  averaged  data, 
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Figure  5.  Computer  wind  component.  extended  with  2  km  averaged  data. 

U.  ANALYSIS  QF  SIMULATED  IMPAfili.  For  th.  .lx  iv.lUblj  Hollomiin 
December  mtt  rntfliAgts,  ont  can  compute  IB  impaota  rtprtMnting  tna  obaarvao, 
1  km,  and  2  km  averaged  data.  Flgura  6a  praaant.  thaaa  . lmulated  impact., 
Th.  Impact,  d.rlvad  from  u.lng  tha  Daoambar  22  avarag.d  profila.  pre. anted 
In  th.  la.t  thra.  flgura.  ara  now  ldantifiad  with  a  6.  Nota  that  tha  dark  6 
rapra.ant.  two  Impact,  and  tha  light  6  rapra.ant*  tha  othar.  Ona  can  ..a 
the  .am.  ganaral  ra.ult.  for  tha  othar  numbarad  impact.,  For  thi.  month  tha 
.imulatad  Impact,  darivad  from  ualng  tha  ob.arvad  and  1  km  avaragad  data  are 
almo.t  idantlcal.  Pairing  tha  impact,  ra.ulting  from  tha  ob.arvad  data  with 
tha  corra. ponding  Impact,  from  tha  1  km  and  2  km  avaragad  data,  ona  can  aae 
a  algnif leant  diffaranca  In  u.lng  tha  propo.ad  zone  thlckna.ae.  a. 
Lllu.tratad  In  flgura  6b.  Not  all  ra.ult.  Include  thl.  ea.lly  derived 
graphical  Interpretation;  therefore,  teat,  for  the  mean  component 
Inference,  were  automated  (Rickm.r.  and  Todd,  1967), 


BIX  SIMULATED  IMPACTS  USING 
0.3,  1.0,  I,  1.0  km  MET  ZONE  THICKNEB8 
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Figure  6a,  Simulated  impacts.  Figure  6b.  Paired  differences. 
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The  first  statistical  test  is  the  Student  t-test.  The  paired 
differences  (n,  average,  and  sigma)  are  used  to  derive  a  t*value 

t  _  A— Sn  6.  /B\ 

computed  <x^  v  ' 


where 

d  -  average  difference  between  pairs 

6  -  hypothesized  pair  difference 

ffd 

crj  -  standard  error  -  ~— 

/n 


In  this  paper  and  in  general  S  is  made  equal  to  0.  By  comparison  with  the 
t- table  value,  one  can  determine  presence  of  a  significant  difference  using 
the  5-percent  alpha  risk.  If  the  oomputed  value  is  within  the  ±  table 
value,  then  one  can  aocept  the  null  hypothesis  that  no  difference  exists. 

Another  test  is  the  F-ratlo  analysis  of  variance,  Theoretically  the  F 
distribution  is  the  square  of  the  Student  t  and  a  conclusion  for  the  two 
tests  is  the  same.  However,  the  F-ratio  analysis  of  variance  provides  other 
Information  that  the  Student  t  does  not.  In  computing  the  different  ratios, 

p  _  moans,-,  , 

variance  for  errors  v  ' 


a  significant  difference  among  the  six  met  rawlnsonde  flights  was  revealed. 
In  the  selected  two-factor  setup  of  the  F-ratio  test,  each  observation  is 
assumed  to  be  determined  by  the  following  four  possible  effects:  the  general 
mean  of  the  data,  a  possible  zone  thickness  effect,  a  possible  atmospheric 
(rawlnsonde  flights)  effect,  and  the  effect  of  the  error  in  our  hypothetical 
mathematical  model  of  the  sum  of  the  variances. 

These  tests  were  performed  on  the  component  impact  differences. 
Analyses  of  the  resulte  were  difficult  to  interpret  because,  as  illustrated 
in  table  3,  the  cross-component  1-km  average  contains  the  computed  t-  and 
F-values  outelde  the  corresponding  table  values.  The  conclusion  is  that  the 
1-km  average  provides  signifioantly  different  cross -component  impacts  than 
those  Impacts  derived  from  the  observed  data,  Different  conclusions  from 
the  individual  comparisons  for  each  component  Impacts  were  not  encouraging. 
The  graphical  results  in  figure  6b  indicate  that  the  observed  and  1-km  zone 
averaged  data  provide  similar  impacts  and  should  not  be  considered  as 
significantly  different,  This  is  certainly  true  when  considering  the  lethal 
radius  of  the  delivered  munition,  To  maintain  agreement  with  the  graphical 
results,  a  third  statistical  test  was  automated. 

2 

The  Hotelling  T  test  uses  the  mean  vector  impact  differences  and 
allows  easier  interpretation  of  the  inference  results.  Table  3  now  reveals 
that  there  is  no  significant  difference  between  the  observed  and  the  1  km 
averaged  data  for  the  December  flights  collected  at  Holloman.  However,  the 
comparison  between  impacts  derived  from  the  observed  and  the  2  km  averaged 
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data  reveal  that  there  la  a  significant  difference.  Under  these  atmospheric 
conditions  and  this  high  quadrant  angle  of  fire,  the  met  data  must  be 
formatted  into  1  km  averages  to  allow  accurate  artillery  fire. 

TABLE  3.  INFERENCE  TEST  RESULTS 


U8ING  HQE  SIMULATED  IMPACTS  FOR  DEC  HOLLOMAN  MET  DATA 

STUDENT  t  F  RATIO  HOTELLING  T* 


TABLE  VALUE 

oroee  range 
2.67  2.67 

oroee 

6.61 

range 

6.61 

vector  (F) 

6.04 

1  km  ZONE 

2.71  -1.66 

7.33 

2.41 

3.77 

2  km  ZONE 

6.00  -7.06 

34.82 

49.83 

20.04 

Not  only  are  the  Hotelling  results  easier  to  Interpret  but  this  test 
also  allows  faster  computations  by  using  subroutines  from  the  IMSL.  No  new 

2 

software  development  or  table  lookup  Is  required.  The  computed  T  value  is 
defined  (Johnson  and  Wiehern,  1989)  as  the  quadratic  form;  and  knowing  the 
mean  and  the  oovarlanoe  matrixes,  one  can  derive  the  result  by  utilizing  the 
IMSL  subroutine,  "blinf." 


where 

X 

(pxl) 

<p8p> 

<p8i> 

p 

X 


quadratic  form  "blinf" 


n  (  X  -  M  )fc  S'1  (  X  •  M  ) 


n 


n 

£  }  <xj  • *> 

J-l 

a  plausible  value 

2  (the  range  and 
the  matrix  of  paired  impact  differences. 


«j  •  x)t 

for  the  mean 
cross  variables) 


(7) 

(8) 
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The  distribution  of  T  has  the  following  F  distribution  relationship: 

(n  ■  1)  p 

(n  -  p)  Fp,n-p 


where 


n  is  the  number  of  paired  impact  differences  and 

p  is  2  representing  the  range  and  cross  variables. 

Solving  for  F  in  the  above  equation,  one  can  compare  the  computed  value 
with  the  table  value  obtained  by  using  the  "fin,"  IMSL  subroutine.  Another 
method  to  determine  the  presence  of  a  significant  difference  is  to  compute 
the  probability  -  1  -  "fdf"  where  "fdf"  (IMSL  subroutine)  is  the  cummulative 
distribution.  The  alpha  risk  is  then  used  as  the  criteria  to  demonstrate 
presents  of  the  null  hypothesis,  no  significant  difference. 

5._ EXPERIMENTAL  RESULTS.  In  general  the  statistical  comparisons  yield 
results  from  the  following  categories:  (1)  during  stable  weather  conditions 
all  proposed  zone  averages  demonstrate  agreement  with  observed  data;  (2) 
during  unstable  weather  conditions  all  proposed  zone  averages  demonstrate 
disagreement  with  observed  data;  and  (3)  under  certain  weather  conditions 
one  zone  average  exists  that  is  a  better  observed  data  estimator  than  the 
other  proposed  zone  average. 

The  Hotelling  results  for  each  month  are  listed  in  table  4.  Beginning 
with  the  Holloman,  December,  high  quadrant  angle  results  which  have  already 
been  dir  sussed  in  the  previous  two  sections,  let's  define  the  "N"  for  no 
.significant  difference  and  the  "Y"  for  a  significant  difference.  By 
reviewing  the  results  in  table  3,  one  can  see  that  the  computed  F  for  the  1- 
km  zone  is  within  the  ±  table  values  and  that  the  computed  F  for  the  2 -km 
zone  is  not  within  the  ±  table  values.  Therefore,  for  the  high  quadrant 
angle  we  note  an  "N"  for  the  l>km  zone  and  a  "Y"  for  the  2 -km  zone  during 
the  month  of  December.  For  February  there  are  only  two  rawinsonde  flights 
(n)  and  since  the  Hotelling  test  requires  that  the  number  of  variables  be 
equal  to  the  degree  of  freedom  (p)  ,  the  (n  -  p)  computation  in  the 
denominator  of  the  Hotelling  distribution  leads  to  a  division  by  zero.  So 
the  other  symbol,  in  the  table  indicates  that  there  is  not  enough  data 
to  complete  the  test.  For  the  other  stations,  may  also  indicate  that 
there  is  no  data  for  completing  any  mean  inference  test. 

Overall,  the  majority  (30  cases)  of  the  Hotelling  mean  vector  inference 
test  of  the  paired  impacts  derived  from  the  met  flights  averaged  into  the 
different  zones  reveal  no  significant  difference.  In  the  minority  (12 
cases),  particularly  for  low  quadrant  angle  of  fire,  none  of  the  proposed 
zone  met  averages  are  representative  of  the  observed  data.  This  observation 
is  true  for  all  months  during  the  year  but  seems  to  occur  more  often  at  the 
SMR  station,  which  is  located  closer  to  complex  terrain.  Eight  of  the 
remaining  11  cases  show  that  the  1-km-zone  averages  yield  better  impact 
results  than  the  2-km-zone  averages.  This  further  demonstrates  that  the 
1-km-zone  averages  produce  almost  identical  simulated  impacts  as  those 
produced  by  using  the  1-min  (-300  m)  observed  met  data. 
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TABLE  4.  SIGNIFICANT  DIFFERENCE  BETWEEN  OBSERVED  AND  PROPOSED  ZONE 


LOW  QUADRANT  ANGLE 

USING  HOL  SIMULATED  IMPACTS  FOR  MONTHLY  MET  DATA 
1  2  3  4  S  S  7  8  9  10  11  12 


1  km  ZONE  N 


NNN  NNYYNNN 


2  km  ZONE  N 
1  km  ZONE  N 


NNY  NNYYNNN 
HIGH  QUADRANT  ANGLE 
NNY  NNNNNNN 


2  km  ZONE  N 


NNY  NNNNNYY 


U8INQ  8MR  SIMULATED  IMPACTS  FOR  MONTHLY  MET  DATA 
LOW  QUADRANT  ANGLE 
1  2  3  4  S  6  7  8  0  10  11  12 


1  km  ZONE  Y 

2  km  ZONE  Y 

1  km  ZONE  N 


Y-Y  Y-YNNN- 
Y-YV-YYYN- 
HIGH  QUADRANT  ANGLE 
n-nn-ynnn- 


2  km  ZONE  N 


N  -  NN-NYN-- 


U8ING  WSD  SIMULATED  IMMCT8  FOR  MONTHLY  MET  DATA 
LOW  QUADRANT  ANGLE 
1  2  S  4  8  S  7  8  0  10  11  12 

1  km  ZONE  Y-Y  -  NNYNYN-" 

2  km  ZONE  Y-Y  -  YNYNYN-- 

HIQH  QUADRANT  ANGLE 

1  km  ZONE  N-N  -  NYNYNN-- 

2  km  ZONE  Y  -  N  -  NNN  N  N  N  -  - 


6.  SUMMARY  AND  RECOMMENDATION.  From  the  majority  of  the  226  rawinsonde 
flights  evaluated,  It  Is  revealed  that  the  atmospheric  state  can  be 
accurately  represented  by  either  1  or  2  km  zone  averages.  Under  this 
condition  the  artillery  may  use  the  already  available  fallout  zone  extension 
to  the  computer  met  message.  However,  this  paper  also  Identifies  cases 
where  the  atmospheric  state  could  not  be  represented  by  any  of  the  proposed 
(1  km  or  2  km)  zone  averages.  Under  this  condition  the  artillery  must  use 
all  available  observation  to  allow  accurate  artillery  fire.  Since  the  1  min 
(-300  m)  would  require  some  33  new  lines  to  extend  the  current  computer  met 
message  to  the  new  height  of  30  km,  the  300-m-zone  thickness  is  not  a 
possible  solution  because  of  the  limited  fire  direction  computer  storage 
constraint.  The  upper  minority  of  cases  reveal  that  the  1  km  is 
representative  of  the  observed  met  data.  This  result  and  the  majority  of 
cases  that  reveal  no  difference  in  zone  thickness  indicate  that  the  1-km 
zone  is  the  optimal  of  the  two  proposed  zone  thicknesses. 


In  conclusion,  it  is  recommended  that  10  new  lines  be  required  to 
describe  the  atmosphero  between  20  and  30  km  above  the  surface.  Future  fire 
direction  computers  shall  have  storage  available  for  37  lines  of  met 
information.  Each  line  shall  contain  an  average  value  for  wind  direction, 
windspeed,  virtual  temperature,  and  pressure  at  the  designated  height.  The 
extended  computer  met  message  zone  structure  for  long-range  artillery 
surface-to-surface  fire  shall  be  composed  of  ten  1-km  zones  averages. 
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ABSTRACT 

The  prediction  of  future  demands  for  each  of  over  200,000  demand-based  repair  parts  is  crucial 
to  the  readiness  of  the  Navy,  and  certainly  other  branches  of  the  Armed  Services.  Item  mana¬ 
gers  with  only  eight  quarters  of  history  are  responsible  for  ordering  that  which  is  sufficient 
to  fill  material  requirements  over  an  average  eight  quarter  procurement  leadtime  and  simul¬ 
taneously  staying  within  economic  bounds  and  excess  material  constraints. 

Pivotal  in  demand  prediction  is  the  ability  to  detenoine  whether  the  demand  time  series  are 
stable  or,  if  trending,  are  they  increasing  or  decreasing. 

Former  methods  do  not  seem  to  do  the  job  now,  and  one  reason  is  that  the  average  procurement 
leadtime  is  nearly  twice  what  it  was  10  years  ago. 

The  present  approach  has  turned  to  the  use  of  nonpar ametric  analysis,  in  particular,  using 
Kendall's  S  (or  T).  The  following  factors  and  levels  are  considered: 

•  Window  Size:  N  “  A,  6,  8,  10  sample  sizes  of  recorded  demand 

•  Demand  Level:  Poisson,  low,  medium,  high,  very  high  average  quarterly  demand  x 

•  Ratio  of  <r/x:  Five  levels  dependent  on  the  level  of  demand 

Extensive  simulation,  as  well  as  actual  demand  data,  are  used  to  estimate  the  degree  of  type  1 
and  type  2  errors  and  the  expected  performance  of  the  "S"  statistic  for  various  combinations 
of  demand  and  variability  levels.  Kendall's  "S"  does  a  good  job  confirming  no  trend  when 
there  is  no  trend.  Also,  it  is  satisfactory  to  detecting  trend  when  there  is  trend  and 
discriminating  between  increasing  and  decreasing  trends . 

The  weakest  link  in  the  chain,  so  to  speak,  is  detecting  the  commencement  of  a  trend  in  demand 
lumediately  following  a  nontrending  period,  as  well  as  Identifying  when  a  trend  has  terminated 
and  the  process  generating  demand  is  now  stable.  A  lag  of  maybe  three  periods  can  occur. 
However,  proper  selection  of  window  length  plays  an  important  role  here  in  minimizing  the  re¬ 
action  time. 


I.  INTRODUCTION.  Prior  to  this  research,  the  Navy  had  been  using  a  trigger- 
trend  statistic  to  identify  situations  where  demand  was  trending;  namely, 


T  -  2  (Dl  +  °z) 

Dj  +  D2  +  D3  +  D4 

where  D1(  D2,  D3,  and  DA  are,  respectively,  the  last  four  quarterly  demands,  Dx 
being  the  most  recent.  Thus,  the  trend  statistic  "T"  was  simply  the  ratio  of 
twice  the  sum  of  the  last  two  quarters  of  demand  compared  to  the  sum  of  the 


21 


Preceding  Page  Blank 


last  four  quarters  of  demand.  The  value  of  this  statistic  could  range  from  a 
minimum  of  zero  to  a  maximum  of  two, 

When  this  statistic's  value  lay  between  ,9  and  1.1,  it  was  concluded  that 
no  appreciable  change  in  the  demand  pattern  had  occurred  and  the  last  demand, 
D1(  was  exponentially  smoothed  with  the  last  forecast  to  compute  a  new  fore* 
cast.  The  nontrending  smoothing  weights  were: 


.1  <DX>  +  .9  (Last  Forecast)  -  New  Forecast 


On  the  other  hand,  when  this  statistic  was  less  than  .9  or  greater  than 
1.1,  it  was  concluded  that  a  trend  existed  and  the  smoothing  weights  were  in* 
creased  to  .3  and  .7,  thereby  placing  more  emphasis  on  recent  demand. 

Experience  over  the  last  decade  showed  an  increase  in  the  percent  of  con¬ 
tracts  for  material  that  had  to  be  cancelled  because  excess  material  was  on- 
hand  or  on- order.  The  counterpart  to  this,  insufficient  material  on-hand  or 
on-order,  was  also  on  the  rise.  Additionally,  procurement  leadtimea  in  the 
past  10  years  have  risen  from  a  year  on  the  average  to  two  years,  significantly 
increasing  the  material  forecast  horizon. 

Both  top  management  and  Individual  item  managers  believed  that  this  trig¬ 
ger  was  overreaotlng,  that  it  was  too  sensitive  to  random  noise  in  the  system. 
This  study  was  undertaken  to  examine  the  trend  detection  method  more  closely, 
Chart  I  portrays  evidence  confirming  these  management  concerns  from  the  quar¬ 
terly  trend  statistics  for  the  period  June  1987  through  March  1989, 


22 


CHART  I 

CUMULATIVE  PCT  ITEMS  DEMAND  TRENDING 
BEFORE  FORECASTING  INITIATIVES 


ooooooooooo 

O  O)  01  Is-*  to  m  -t  co  w  r 
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SOURCE.  SPCC  MAR '89  LEVELS  STATS 


Our  first  examination  of  formula  (1)  was  done  by  using  a  table  of  random 
numbers  to  obtain  a  sample  frequency  distribution  of  the  trend  statistic.  The 
demands  were  separated  into  three  categories;  0  to  1,  1  to  5,  and  6  to  9  units 
of  quarterly  demand, 

Using  a  sample  of  size  40,  we  obtained  for  the  random  nontrending  (low 
usage  rates)  demand  of  size  0  or  1  the  following  pattern  of  values  for  the 
statistic; 


Trend  Statistic 

0 

.67 

1.00 

1.33 

2.00 

Frequency 

3 

3 

19 

4 

9 

It  is  easily  seen  that  using  trigger  values  of  ,9  and  1.1  would  conclude 
that  demand  was  trending  S3X  of  the  time  for  low  demand  items.  Furthermore, 
it  is  interesting  to  consider  the  16  possible  permutations  of  Os  and  Is  over 
the  eight  quarters.  Here  we  find  the  values  for  the  trend  statistic  to  be: 


Trend  Statlstlo 

0 

2/3 

1 

4/3 

2 

Frequency 

3 

2 

3 

2 

4 

So  much  depends  on  the  case  of  four  oonseoutlve  periods  of  aero  demand. 
This  situation  has  no  numerical  value,  but  oommon  sense  would  indicate  that  we 
should  assign  these  cases  the  value  of  one,  If  this  is  done,  the  statistic  is 
biased  to  the  right  of  one,  Also,  it  is  interesting  to  note  what  peroentage 
would  be  declared  trending  using  different  lower  and  upper  bounds  on  the  trig* 
get  statistic, 

Trigger  Statlstlo 

Ihruhflld  Bounds .  firaant  Xilggmd 

Low  High 

.9  to  1.1  70X 

.8  to  1.2  70X 

.7  to  1.3  70X 

.6  to  1.4  43X 

Thus,  the  trigger  trend  statistic  %'ould  still  indicate  trend  in  43X  of  the 
oases  even  using  for  a  lower  bound  any  number  from  0  to  ,6,  and  for  the  upper 
bound,  any  value  from  1.4  to  2.  At  this  point,  it  became  clear  that  no  sta¬ 
tistic  such  as  this  should  be  used  on  a  string  of  zeros  and  ones.  Yet,  the 
majority  of  items  managed  by  the  Navy  have  characteristically  low  demand 
rates . 

A  similar  analysis  was  conducted  for  the  medium  demand  category  (one  to 
five  demands  per  quarter)  using  a  random  sample  of  size  20.  The  following 
table  was  obtained  for  the  same  statistic  (values  were  rounded  off  to  the 
nearest  tenth) ; 
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Trend  Statistic 

.3 

.6 

.8 

1.0 

1.1 

1.3 

1.4 

1.6 

Frequency 

1 

3 

3 

4 

1 

5 

1 

2 

Again,  the  mean  ia  allghtly  larger  than  one  and  the  present  lower  and 
upper  bounds  on  the  trigger  (.9,  1.1)  would  signal  trend  on  the  lower  side 
about  3SX  of  the  tins  and  about  40X  on  the  upper  side.  A  simple  examination 
of  the  data  suggested  changing  the  thresholds  to  .6  and  1.6  for  low  and  high 
cutoffs  to  obtain  80X  assurance  that  a  trend  ia  not  declared  for  common  sta- 
tionary  demand  situations. 

Finally,  for  the  high  demand  (values  of  six  to  nine  demands  per  quarter) 
using  a  random  sample  of  size  30,  we  found  the  following  values: 


Value 

.87 

.90 

.91 

,93 

.94 

.96 

.97 

1,00 

1,03 

1.04 

1.07 

1.13 

1,14 

Frequency 

1 

1 

2 

3 

2 

3 

6 

4 

4 

1 

1 

1 

1 

For  high  demand  items,  it  appears  that  the  trend  statistic  becomes  more 
stable  and  lies  within  a  decreased  computational  range.  The  threshold  values 
(.9,  1.1)  hardly  come  into  play.  So  in  these  situations,  the  probability  of  a 
Type  I  error  is  relatively  small  and  may  be  tolerable. 

In  September  1989,  action  was  taken  to  widen  the  trigger  trend  threshold 
values  from  (.9,  1.1)  to  (.6,  1,4)  In  an  effort  to  reduce  the  Type  I  error, 
CHART  II  details  the  reduction  in  the  number  of  items  designated  as  trending 
after  these  changes  were  accomplished.  Significant  reductions  were  achieved 
for  high  and  medium  demand  items,  but  low  demand  items  were  relatively  un¬ 
affected,  experiencing  almost  a  flat  (45,000  to  50,000  items)  trend  declara¬ 
tion  level.  This  finding  is  consistent  with  our  theoretical  conclusions  dis¬ 
cussed  earlier  relative  to  the  trigger  trend  statietlc  expected  performance 
for  low  demand  items  with  a  large  percentage  of  zero  observations. 

This  relatively  minor  forecasting  change,  based  only  on  straightforward 
extensions  of  probability  theory  and  operations  analysis  techniques,  led  to  an 
immediate  10  million  dollar  per  quarter  reduction  in  inventory  requirements 
changes.  Requirements  changes  (churn)  due  to  quarterly  reforecasting  of  de¬ 
mand  was  measured  in  June  1989  as  approximately  30  million  dollars  per  quarter 
at  SFCC.  Of  this  amount,  10  million  dollars  worth  of  changes  was  attributed 
directly  to  the  prior  overly  sensltivo  trend  trigger  thresholde  of  (.9,  1.1) 
Invoking  (.3,  .7)  smoothing  on  about  11,000  high  and  medium  demand  itema. 

These  reductions  wore  primarily  due  to  decreases  in  the  amount  of  quarter 
to  quarter  change  in  the  key  inventory  requirements  areas  of  economic  order 
quantity  and  reorder  point  computations,  Operationally,  whenever  unnecessary 
or  unwarranted  requirements  changes  are  reduced,  better  estimates  of  future 
needs  and  budgut  requirements  can  be  made  and  rework  of  procurement  orders  are 
reduced.  Likewise,  significant  reductions  can  be  achieved  in  excess  material 
on-order  as  forecasts  become  more  stable. 
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fi'i-m  the  theoretical  aspect,  we  recognize  in  formula  (1)  that  by  dividing 
the  numerator  and  denominator  of  tho  statistic  by  four,  that  it  is  essentially 
a  quotient  of  the  means  of  a  sample  of  size  two  and  a  sample  of  size  four. 
Furthermore,  the  two  means  are  correlated  in  that  the  two  most  recent  quarters 
of  demand  appear  in  both  the  numerate,  and  denominator. 

Much  has  been  written  on  the  product  and  quotient  of  two  random  variables, 
It  is  a  theoretically  complicated  subject.  Only  in  the  restricted  situation 
of  two  normally  distributed  variables  that  are  correlated  has  an  explicit 
density  function  been  derived.  In  this  case,  E.  C.  Fialler  in  1932  showed 
that  if  x  and  y  are  bivariate  normal  with  correlation  p ,  the  statistic 
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has  frequency  function 
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It  doesn't  seem  defensible  to  try  to  invoke  such  theory  in  our  present  situs* 
tlon.  Now  that  it  was  clear  that  the  previous  method  used  to  detect  trends 
was  problematic,  our  attention  and  research  turned  to  alternative  methods  for 
trend  detection  and  estimation. 
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II.  FIRST  ATTEMPTS  TO  IMPROVE,  Having  concluded  Chat  the  trigger 'trend 
statistic  had  serious  operational  Limitations  (poor  trend  detection  and  poor 
stability  confirmation)  we  began  research  into  alternative  methods  for  trend 
detection  in  time  series  data.  Our  goal  being  to  select  and  evaluate  a  method 
which  was: 

•  Simple  to  use  and  Interpret  by  Inventory  Managers. 

•  Robust. 

•  Invariant  to  distribution  assumptions. 

•  Excellent  ability  to  oonfirm  stationary  demand. 

•  Relatively  good  ability  to  detect  substantial  trends  and  process 
change . 

•  Could  be  used  effectively  with  small  sample  sices. 

Ve  sought  first  to  identify  a  suitable  method  to  detect  trends  and  then  sub* 
sequently  select  a  method  to  ascertain  the  level  of  trend  present. 

The  work  of  Glnl  provided  a  fundamental  understanding  of  the  relationship 
of  the  variance  between  Individual  observations  and  the  variance  of  the  series 
of  observations.  From  this  research,  we  come  to  the  common  sense,  but  much 
overlooked  conclusion,  that  for  a  nontrending  time  aeries,  eaoh  of  the  obser¬ 
vations  are  equally  likely  events.  That  the  highest  and  lowest  values  may  be 
first,  last,  or  in  the  middle  of  the  series  and  yet  no  trend  may  be  present. 

Tintner'a  work  on  the  variate  difference  method  provided  an  Important  in¬ 
sight  and  understanding  to  our  most  prevalent  demand  pattern  , , ,  the  ZIG-ZAO 
demand  pattern.  From  his  work,  we  were  able  for  the  first  time  to  realise 
that  the  majority  of  our  items  were  not  trending  but  rather  osolllating  be¬ 
tween  values  of  high  to  low  observations.  Many  trend  detection  methods  ex¬ 
perience  difficulty  in  handling  oscillating  demand  patterns  and,  in  particu¬ 
lar,  our  trend- trigger  statistic  is  overly  sensitive  to  such  common  demand 
relationships. 

Closely  related  to  all  this  is  what  we  oall  the  mean  squared  successive 
differences.  Here  we  compute  the  average  of  the  squares  of  the  (n-1)  success¬ 
ive  differences  between  successive  elements  in  a  random  sample  of  slse  n.  It 
has  been  shown  that  the  expeoted  value  of  this  statistic,  namely  of 


*  («l»i  '  »i)* 

n-1 


(2) 


is  2ff*,  when  the  base  population  distribution  is  normal.  But  the  expected 
value  of  the  ordinary  sample  variance,  namely  of 
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a  g  <*1  "  »>* 

n-1 


is  a\,  Thera fora,  wa  can  aay  the  ratio 


haa  axpaotad  value  2. 

Dr.  von  Neumann  gave  ua  the  dlatrlbutlon  function  of  f ?,  and  In  1942  Dr. 
Hart  provided  a  table  of  ita  valuaa.  However,  aa  haa  happened  in  ao  many 
other  aituatlona,  Dr.  von  Neumann  aaaumed  the  aampla  came  from  a  normal  dla¬ 
trlbutlon.  So,  In  order  to  uae  thla  atatiatio,  we  are  parametrically  bound. 
When  data  haa  an  upward  trend,  ia  will  increaae  much  leaa  than  aa,  ao  i)  would 
be  leaa  than  2.  On  the  other  hand,  if  the  data  rapidly  goea  up  and  down  (a 
altuatlon  which  we  now  face  to  which  our  preaent  trigger  aeaaa  to  be  senal- 
tlve) ,  fia  will  Increaae  proportionally  greater  than  a*.  Then  r\  will  be  grea¬ 
ter  than  2  and  would  be  unaatiafactory. 

Some  attention  waa  alao  glvan  to  ualng  a  algn  teat  or  a  runa  teat  ualng 
the  median.  The  analyala  ahowed  auch  examination  would  not  detect  nonrandom- 
neaa  when  t;  did. 


III.  TREND  DETECTION  USING  SANE  CORRELATION  APPROACH.  In  thla  work,  it  ia 
difficult,  if  not  impoaalble,  to  defend  any  aeaumptlon  of  normality,  A  mea¬ 
sure  which  la  robuat  la  dealred,  Kendall 'a  S  (Tau)  in  based  on  the  renka  of 
obaervatlona  aa  oppoaad  to  the  magnitudes  associated  with  them.  What  la  im¬ 
portant  la  that  the  "SH  dlatrlbutlon  la  independent  of  the  dietrlbutiona  of 
the  varlablea  of  interest,  a.g.,  the  independent  and  dependent  variable  in 
linear  regression,  Interesting  and  useful  ia  the  fact  that  if  two  variables 
have  a  monotonic  relationship,  their  ranks  will  have  a  regression  that  la 
apparently  linear.  Hence,  the  ranks  can  be  considered  to  be  transformed 
variables . 

Let  ua  now  discuss  the  concept  of  rank  correlation  as  it  applies  in  our 
case  to  demand  observations  over  a  period  of  time.  Consider  a  sat  of  data 
values  ordered  in  time;  i.e.,  demand  observations  ranked  by  quarter: 


where 

<*i.  yii  *m. 
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(time  in  quarters) 

y  -  A 

B  C  D  E 

F 

c 

H 

(observed  values) 
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If  there  is  a  trend,  there  will  appear  to  be  an  arrangement  or  ordering  to 
the  data.  That  is,  if  the  ranks  are  correlated,  then  as  "x"  increases  "y" 
will  generally  increase  or  decrease.  Where  there  is  no  trend,  we  should  ex- 
peot  the  values  to  be  randomly  arranged.  Kendall  refers  to  the  first  case  as 
"array"  and  the  second  as  "disarray"  in  the  data. 

Stan: 

(1)  Let  us  first  arrange  one  rank  In  its  natural  increasing  order: 

Time  -  xx,  xa,  X*  .  .  , 


Then  we  compare  all  possible  [a]  demand  observation  values  two  at  a 
time.  For  example,  for  the  first  paired  observations: 

where  yx  -  A  and  ya  -  B 

(2)  Upon  inspection,  if  we  find  that  ya  >  yx,  implying  that  B  >  A,  then 
Kendall  would  call  the  pair  concordant,  sinoe  the  values  are  in  agree¬ 
ment  or  arrayed  with  respect  to  the  ordering  of  time.  For  eaoh  auoh 
pair  assign  a  score  of  +1. 

(3)  However,  if  we  find  that  ya  <  yx,  implying  that  B  <  A,  then  the  pair 

is  oalled  disoordant,  sinoe  the  values  are  in  disagreement  or  disarray 
with  respect  to  time.  For  each  suoh  pair,  assign  a  score  of  -1. 

(4)  In  the  special  case  where  ya  -  yx,  that  is  B  -  A,  the  pair  is  consid¬ 
ered  to  be  tied  in  ranks  and  a  score  of  zero  is  assigned. 

(5)  The  Kendall  "S"  statistic  is  simply  the  sum  of  the  number  of  concor¬ 
dant  pairs  less  the  sum  of  the  number  of  disoordant  pairs,  It  is  a 
measure  of  the  rank  correlation  between  the  order  of  the  variables  in 
time  and  their  order  in  magnitude.  The  following  table  displays  the 
probabilities  that  the  absolute  value  of  "S"  attains  or  exceeds  a 
specified  value  for  a  sample  size  of  eight  observations; 

KENDALL  "8"  PROBABILITIES 
SAMPLE  SIZE  OP  S  OBS 

•  PROBABILITY  "S"  ATTAINS  OR  EXCEEDS  A  SPECIFIED  VALUE 
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The  Kendall  "S"  statistic  measures  the  degree  of  rank  correlation  on 
ordering  present  In  the  data.  Its  range  Is  equal  to  the  maximum 
number  of  pairs  of  comparisons  that  can  be  found  between  the  two 
rankings.  In  the  present  case,  "S"  will  range  from  (-28,  +28): 


[l]  -  aiaiU  .  .  28 

it  it 


In  this  case,  if  we  assume  a  nontrending  stationary  demand  time  series 
with  28  possible  rank  combinations,  one  would  expect  from  theory  that 
exactly  14  pairs  would  be  concordant  and  14  will  be  discordant  yield¬ 
ing  ("S "-+14-14-0)  an  "SH  value  of  zero.  Inspection  of  the  above 
table  would  suggest  that  this  condition  would  occur  about  S5X  of  the 
time. 

Where  the  data  is  perfectly  arranged  In  increasing  or  decreasing 
order,  "S"  would  attain  values  of  (+28,  -28),  respectively.  Between 
these  extremes  of  perfect  to  zero  rank  correlation  probabilities  can 
be  assigned  for  observed  values  of  "S"  which  can  form  the  basis  for 
hypothesis  testing  against  the  existence  of  trend. 

(6)  Construct  an  hypothesis  test  against  the  existence  of  trend. 

-  H0i  No  trend  exists 
•  Hx :  Trend  is  present  in  the  data 

We  will  reject  the  hypothesis  He  that  no  trend  is  present  in  the  data 
if  the  probability  of  "S"  equals  or  exceeds  a  specified  critical 
value.  In  the  table  above,  we  selected  a  critical  value  of  about 
seven  percent  yielding  an  "SH  value  of  13  which  must  be  achieved  or 
exceeded  before  we  would  reject  the  hypothesis  of  no  trend  present  in 
the  data. 

For  the  reader  not  familiar  with  Kendall's  S  (Tau)  in  trend  analysis,  we 
present  a  numerical  example: 

NAVY  PART  NUN  00-025-3457 
Demand  12232372 
Quarter  12345676 

Oldtifc  Labwt 

Rule :  Subtract  every  demand  from  each  one  that  precedes  it,  If  great¬ 

er,  assign  +1;  if  less,  assign  -1.  If  equal,  acslgn  0.  Sum  the 
positive  and  negative  scores  to  calculate  "S". 
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Owen's  Tables 


Kendall's  Table 


Thus,  Kendall's  "S"  with  an  error  probability  of  about  7%  suggests  a  high 
probability  of  an  upward  trend  present  In  the  data.  Kendall 'a  Tau  (a)  like¬ 
wise,  Is  a  simple  linear  transformation  of  HS"  and  Is  oaloulated  by  dividing 
"S"  by  28.  With  an  effective  operating  range  of  (-1,  +1),  Kendall’s  Tau  (a) 
parallels  the  range  of  the  Pearson  correlation  coefficient  of  parametric  sta¬ 
tistics. 


IV.  NONPARAMKTRIC  TUNS  K8TDUTX0N.  It  Is  well  known  that  an  outlying 
observation  can  have  an  appreciable  effect  on  the  position  of  the  parametric 
least  squares  fit.  On  the  other  hand,  the  regression  that  passes  through  the 
sample  median  of  X  and,  simultaneously,  the  sample  median  of  Y  instead  of  the 
centroid,  Is  less  sensitive  to  outlier  observations.  If,  In  addition  follow¬ 
ing  Sen,  we  use  for  the  slope  the  median  of  the  set  of  slopes  determined  from 
all  possible  pairs  of  points,  we  will  have  induced  more  stability.  For  "n" 
data  points,  the  slope  set  will  have  n(n-l)/2  slopes  and  this  seems  to  have 
something  in  common  with  the  calculation  of  Kendall's  S  (Tau)  to  detect  trend. 
It  is  Interesting  to  note  that  the  classical  least  squares  estimate  of  slope 
can  be  obtained  from  a  weighted  average  of  these  individual  slopes  as  can  be 
done  for  the  Intercept. 

The  28  slopes  have  a  median  of  .29,  while  the  least  squares  estimate  is 
.40.  The  centroid  is  (4.5,  2.75)  while  the  median  point  is  (4.5,  2).  It  Is 
simply  accidental  that  both  Intercepts  equal  .93. 

From  Sen  we  have  N  -  21  (nonzero),  n  -  8,  (S)  -  28  and  using  Kendall's  <n 
-  .054  and  U*  -  14/28,  we  calculate 


N*  -  J  21  (!)  |  *  2?  "  l2’12 


<3) 
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Th*n,  the  lovar  and  uppar  90X  rank  confidences  values  are 


Mx  -  j  (21  -  12.12)  -  4.44  -  4 


Ma  -  j  (21  +  12.12)  -  16.56  -  17 


<4> 


(5) 


where  we  rounded  down  on  the  lower  rank  and  up  on  the  upper  rank,  as  recom¬ 
mended  by  Conover.  So  the  4th  ranked  slope  and  the  17th  ranked  slope  gives  us 
our  90%  two-sided  confidence  interval  of  the  slope;  i.e,,  (*.25,  .5). 

The  following  graph  provides  a  comparison  of  classical  least  squares  re¬ 
gression  and  the  nonparametric  methods  in  fitting  the  time  series  data  from 
our  example.  It  la  clear  from  the  graph  that  a  positive  trend  component 
exists  in  the  data;  however,  the  median  regression  method  provides  a  more  con¬ 
servative  trend  estimate.  Thus,  this  method  is  less  sensitive  to  the  one  out¬ 
lier  observation  sustained  in  peviod  seven,  This  attribute  may  be  of  partial 
significance  in  those  situations  where  the  trend  component  effect  is  used  to 
compute  future  average  demand  values  eight  or  more  periods  into  the  future  as 
is  the  case  with  Navy  inventory  control  systems. 
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LEAST  SQUARES  vs  MEDIAN  REGRESSION 


V.  EXPERIMENTAL  DESIGN,  Our  overall  objective  was  to  evaluate  the  per¬ 
formance  of  the  Kendall  "8"  trend  test  across  a  broad  range  of  expected  de¬ 
mand,  variability  and  trend  levels,  while  maintaining  control  of  key  variate 
factors  to  ensure  the  results  reflected  intrinsic  expected  model  behavior/ 
performance.  Specifically,  we  sought  to  access  the  ability  of  the  Kendall  nS" 
statistic  to: 

•  Detect  demand  trends. 

•  Transition  effectively  between  alternating  trend/no  trend  periods. 

•  Confirm  demand  stability  in  the  absence  of  trend. 

•  Determine  the  averege  lag  and  delay  in  Identifying  trend  commencement/ 
trend  termination. 

To  maintain  maximum  control  of  the  key  design  factors,  we  initially  chose 
simulation.  By  using  simulated  data  we  could  specify  certain  factors  and 
levels  precisely: 

•  The  underlying  mean  demand  of  the  base  population. 

•  The  degree  of  variability  about  the  mean. 

s  When  a  trend  was  introduced  and  its  level/magnitude. 

s  The  random  variable  noise  level  present  in  the  data  via  the  standard 
deviation  to  mean  ratio. 

These  types  of  factors  and  levels  usually  affect  the  ability  to  deteot  the 
presence  of  a  trend  or  to  confirm  the  lack  of  a  trend  (demand  stability) . 

Our  trend  generating  function  is: 


New  Mean  Demand  -  [Mean  Demand  at  Beginning  of  Trend] 

x  [1  +  (Period  Multiplier  x  (Period  Into  Trend)'"10*  *■"»>] 

"Alpha"  "Bata" 


In  algebraic  symbols  this  is 


y«  -  y«  U  +  «<*>#] 


The  sign  of  a  determines  the  direction  end  magnitude  of  the  expected  equipment 
population  increases  or  decreases.  The  Beta  exponent  or  period  power  is  used 
to  adjust  the  rate  of  increase  or  decrease  in  an  Item's  individual  expected 
failure  rate  per  Installed  population. 
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Each  item  started  with  a  stable  demand  period  -  initialization  phase  - 
followed  by  two  periods  of  trend,  first  t  moderate  trend  and  then  a  stronger 
trend  to  reflect  Increasing  population  and  failure  rate  •  a  growth  phase. 

The  trend  was  then  turned  off  to  represent  the  "top  of  the  ramp"  steady  state 
typical  middle  phase  of  the  system  support  life  cycle.  Next  we  Induced  a 
strong  negative  trend  to  reflect  the  effect  of  de* installations;  i.e.,  popu¬ 
lation  "decline  phase".  Finally,  the  above  decline  was  followed  by  a  stable 
"final  residual  support  phase"  period. 

For  medium,  high,  and  very  high  demand,  wa  used: 

Ferloda  1-8  no  trend 

Periods  9-17  u  -  .05,  $  -  1.5 

Periods  18-25  o  -  .3,  0  -  .8 
Periods  26  -  33  no  trend 

Periods  34-41  a  -  -.02,  0  -  1.5 
Periods  42  •  50  no  trend 

The  following  graph  shows  the  general  shape  of  such  a  demand  activity.  We 
need  to  explain  next  how  we  induced  random  variation  -  noise. 
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Wo  generated  two  random  numbers  from  a  uniform  distribution  on  (0,1). 
Call  them  Rx  and  Ra,  Then  by  M.  Muller's  method,  we  calculated 


V  -  (-2  log  Rt)*9  x  (Oos  2  n  Ra) 


for  normal  deviation  on  (0,1).  To  oonvert  V  to  a  normal  distributed  variable 
with  mean  n  and  standard  deviation  o,  we  used  the  transform 


Va  -  V  x  o  +  n 


Poisson  random  variable  noise  was  generated  using  Martin's  method  based  on 
a  method  suggested  by  Kahn.  Briefly,  one  begins  by  generating  N  uniform  ran* 
dom  numbers  and  successively  multiplying  them  until  the  following  lnequal* 
lty  is  satisfied: 


N 

n  ^ 

l 


Then  (N-l)  la  the  desired  Poisson  random  variable  value  with  a  mean  of  np, 

When  the  first  random  variable  number  generated  satisfies  the  inequality,  then 
the  Poisson  variable  is  assigned  the  value  cero, 

In  the  previous  Seotlon  111,  we  illustrated  the  calculation  of  Kendall's  S 
(Tau)  for  a  window  of  (sample  size)  8.  It  is  apparent  this  statistic  can  be 
oaloulated  for  other  size  windows  and  we  felt  it  necessary  to  examine  the  re* 
action  of  this  statistic  to  various  size  windows.  Obviously,  a  longer  window 
is  not  as  susceptible  to  change  as  a  shorter  window.  As  will  be  seen,  we  will 
take  our  known  generated  pattern  and  utilize  various  window  lengths  on  such  a 
time  series  by  dropping  off  the  earliest  number  in  the  window  and  adding  on 
the  latest  one.  We  will  do  this  for  window  sizes  of  A,  6,  8,  10,  and  12. 


The  second  factor  for  whioh  we  seek  the  effects  at  various  levels  comes 
from  scientific  Judgment  and  experience  in  this  field  of  work.  We  have  seen 
patterns  or  the  lack  of  such  according  to  the  volume  of  demand  for  a  part, 
Consequently,  we  have  classified  demand  into  five  categories  or  levels:  very 
low,  low,  medium,  high,  and  very  high,  given  by; 


Bimind  Catagpry 

Very  low  demand 
Low  demand 
Medium  demand 
High  demand 
Very  high  demand 


Avirigi  Hamad 

.3  to  4  per  year 
1  per  quarter 
3  per  quarter 
9  per  quarter 
20  per  quarter 
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The  third  factor  of  concern  la  demand  variability  which  we  need  to  sirau* 
late  and  control.  After  much  analysis  of  actual  data,  it  was  decided  to  use 
for  the  maaaure  the  ratio  of  the  average  variability  to  the  average  demand, 
the  standard  deviation  to  mean  ratio  or  coefficient  of  variation, 

The  aeleotion  of  levels  for  this  factor  appeared  not  to  be  independent  of 
the  levels  of  the  aeoond  factor.  A  sort  of  Pareto  distribution  analysis  of 
over  9000  randomly  selected  parts  gave  us  the  following  two-way  table; 

Variability  Level 


Demand  Category 

XL 

LQ 

Hfi 

HI 

XH1 

VL 

.49 

.64 

.71 

1.00 

1.60 

LO 

.40 

,70 

.90 

1.21 

1.26 

MD 

.16 

.30 

.55 

.91 

1.13 

HI 

.14 

.28 

.53 

.93 

1.12 

VH1 

.14 

.28 

.53 

.93 

1.06 

VI.  FINDINGS  -  RESULTS  -  RECOMMENDATIONS.  The  50  various  data  sets  used  in 
the  study  each  consisted  of  50  quarters  of  demand;  that  is,  for  each  combina¬ 
tion  of  demand  and  variability,  a  simulated  demand  data  set  was  generated.  In 
addition,  this  was  done  for  Case  <1)  -  a  set  with  three  periods  of  trending/no 
trend  and  Case  (2)  •  a  set  without  trends  (stationary) .  TABLES  1  -  111  pro¬ 
vide  graphs  of  the  simulated  demand  observation  data  sets  used  in  the  study. 
The  Kendall  "S"  test  was  applied  to  eaoh  suoeessive  set  of  observations  in  the 
window  (moving  through  time)  in  an  attempt  to  identify  trending  and  nontrend¬ 
ing  processes, 
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NORMAL  DISTRIBUTION 
NON-TRENDING  DATA  SETS 


'  “*'*'**  **j  '  ‘  •  «  »  »  «  «  • 

mm  ••  *trocp  -  «wep'  •  ewtwMw  ...  mm  -.«trioor»  —  «»idop» 


TABLE  I 


40 


NORMAL  DISTRIBUTION 
TRENDING  DATA  SETS 


VHTR-1-2  qju  =  1.12 


VHTR-3-2  alu  =  -S3 


TABLE  II 
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>  <  OC  —  <zOUJ 


POISSON  DISTRIBUTED  DEMAND 
STATIONARY  DEMAND  TRENDING  DEMAND 


PDNT-1  ;  MEAN  =  0.60 


PDTR-1 


all u  =  2.7 


i  %  7  t  it  it  ir  it  a 

YEARS 

PDNT-2  ;  MEAN  =  1.00 


oljJt  =  2.0 


1  0  »  7  0  11  IS  U  17  10  01  >1  03 

YEARS 

PDNT-3  ;  MEAN  =  1.96 


a/jj  =  1.4 


1  3  I  7  •  II  13  11  17  It  01  » 

YEARS 


PDNT-4  ;  MEAN  =  2.96 


o/fj  =  1.0 


i  t  t  r  t  ii  it  it  17  it  >i  e  a 
YEARS 

PDNT-5  ;  MEAN  =  4,12 


1  0  |  7  0  If  IS  ft  17  10  >1  33  a 

YEARS 


a  In  = 


TABLE 


1  0  |  7  0  11  IS  II  17  10  31  Zi 

YEARS 

PDTR-2 


3  I  7  0  II  13  IS  17  10  01  » 


PDNT-3 


3  I  7  •  11  13  IS  17  10  tl  S3 

YEARS 


PDTR-4 


3  I  7  0  11  13  II  17  it  »i  a 

YEARS 


PDTR-5 


3  I  7  0  II  13  II  17  It  01  03  01 

YEARS 
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For  each  observation  set  within  the  moving  window  across  the  50  data 
points,  we  counted  how  often  the  Kendall  HS”  oorreotly  identified  the  pre¬ 
sence  of  trend  and  how  often  it  oorreotly  identified  the  leak  of  trend;  i.e., 
confirmed  stability. 

The  evaluation  to  be  used  follows  the  following  example  for  eaoh  window 
size  aoross  the  50  data  points. 


ACTUALITY 


TREND 

NO  TREND 

p 

TREND 

12 

1 

R 

E 

NO  TREND 

13 

19 

D 

I 

TOTAL 

25 

20 

C 

T 

SCORE 

48X 

95X 

We  oall  a  Type  1  error  the  classifying  of  aotual  nontrending  as  trending 
and  a  Type  II  error  the  classifying  of  trending  as  nontrending.  In  the  above 
table,  we  predicted  19  of  the  20  no  trending  situations  which  means  the  pro¬ 
bability  of  a  Type  I  error  is  5%;  whereas,  of  the  25  trending  situations  we 
predicted  only  12,  thereby  having  a  probability  of  a  Type  II  erwor  of  52X. 

So  our  immediate  goal  is  to  determine  optimal  window  sizes  in  order  to 
minimize  Type  I  and  Type  II  errors.  Initially,  we  set  performance  goals: 

(1)  Maintain  very  high  capability  to  confirm  stability  for 
stationary  demand  patterns  -  more  than  88X  of  the  time. 

(2)  Maintain  relatively  high  capability  to  detect  that  trend 
Influence  has  ceased  •  more  than  BOX  of  the  time. 

(3)  Select  window  size  maximizing  trend  detection  while  meeting 
(1)  *nd  (2) . 

Practically  this  led  us  to  minimise  Type  II  errors  for  a  specified  Type  I 
goal.  Now  In  using  the  Kendall  "S",  we  tried  to  hold  to  cutoff  valuea  that 
were  exceeded  about  5X.  However,  tabular  values  closest  to  5X  had  to  be  used. 
These  are  given  at  the  bottom  of  eaoh  column  in  the  following  tables. 

The  following  five  tables  give  the  percentages  that  Kendall's  "S"  achieved 
in  being  correct.  Entries  that  are  shadowed  appear  best  for  that  window  size, 
For  example,  look  at  the  table  for  very  high  demand.  Note  the  situation  ap¬ 
parently  improves  as  you  go  from  right  to  left  and  from  top  to  bottom.  But  it 
must  be  remembered  that  for  rows  above  the  last  one,  it  is  best  to  use  a  wider 
window;  i.e.  ,  larger  sample  size  for  the  time  series. 
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The  reader  will  note  that  observations  like  those  in  the  previous  para¬ 
graph  are  not  as  easily  made  for  lower  demand  items.  For  the  low  demand 
items ,  it  appears  we  oannot  come  close  to  our  goal  for  detecting  trend.  The 
same  can  be  said  for  the  very  alow  Poisson  case. 

The  sixth  table  to  follow,  "Overall  Performance",  comprises  the  five  pre- 
ceeding  tables  by  washing  out  window  length  and  aigma/mu  categories. 
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SIGMA/HU 


KENDALL  "S"  PERFORMANCE  ANALYSIS 
MEDIUM  DEMAND  ITEMS  MU  =  3/QTR 


READ  PCT  CORRECT:  TRending,  Non  Trending,  STationary 


KENDALL  "8"  RESULTS 
OVERALL  PERFORMANCE 


SAm,EI 

HIGH 

MEDIUM 

m 

msm 

T 

40  -  75% 

20  -  40% 

20  -  50% 

30  SOX 

NT 

80  *  93% 

80  •  93% 

90  •  100% 

80  -  90X 

ST 

88  •  95% 

•  fi 

88  •  100% 

88  -  100% 

85  -  90% 

"T"  -  TRENDING  DATA  SET 

"NT"  7  NONTRENDING  BUT  PREVIOUSLY  TRENDING 

t 

"ST"  -  STATIONARY  DATA  SETS 

PERCENT  IDENTIFIED  CORRECTLY 
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VII.  THE  IAO.  The  next  five  pages  use  the  same  set  of  data  we  used  earlier 
for  high  demand,  trending  and  not  trending,  and  low  variability,  symbolized 
HDTR-4.  On  each  computer  printout,  the  50  demand  observations  are  blocked  in 
five  blocks,  aa  the  induced  demand  goes  from  no  trend  to  trend  (1),  from  trend 
(1)  to  trend  (2),  from  trend  (2)  to  no  trend,  from  no  trend  to  trend  (3),  and 
finally  to  no  trend.  Then,  using  an  S  with  a  email  probability  -  around  7X  • 
we  followed  S  values  in  an  interval  and  across  intervals. 

For  example,  the  first  page  for  a  window  of  size  4  shows  the  five  calcu¬ 
lated  values  of  S  for  the  eight  values  in  the  beginning  no  trend  period  indi¬ 
cate  correctly  no  trend.  On  the  other  hand,  starting  with  period  9  and 
through  period  17,  when  we  have  a  trend,  the  S  statistic  fails  to  pick  up 
trend  throughout  the  entire  period.  Moreover,  for  four  periods  into  the  third 
stage,  the  S  statistic  still  falls  to  suggest  trend.  Finally,  at  period  22, 
the  S  statistic  indicates  a  trend  and  correctly  predicts  through  period  23. 

We  have  indicated  with  a  dash  (-)  when  S  is  incorrect  and  with  a  star  (*)  when 
it  is  on  target.  For  this  particular  run  of  50  demands,  we  see  nine  correct 
calls  out  of  a  possible  25  for  trending.  However,  out  of  22  nontrending 
cases,  our  statistic  S  is  correct  21  times,  and  so,  is  95X  correct,  or  if  you 
will,  has  a  probability  of  a  Type  Z  error  of  5X.  On  the  other  hand,  our  sta¬ 
tistics  seleoted  only  36X  of  trending  cases;  hence,  we  face  a  sample  probabi¬ 
lity  of  a  Type  II  error  of  64X. 

A  similar  explanation  oan  be  given  for  the  other  four  cases  of  window 
lengths  6,  8,  10  and  12.  It  is  apparent  that  wlnaow  size  of  6  or  8  does 
better. 

This  delay  or  lag  puts  limitations  on  successful  use  of  the  S  statistic 
for  forecasting  as  summarized  in  the  chart  following  the  five  computer  print¬ 
outs. 

It  seems  we  are  faced  with  a  problem  which  lies  within  that  general  pur¬ 
view  described  by  Bennett  and  Franklin  on  page  688  entitled  "Choice  of  Tests 
for  Non- Randomness" . 

It  will  be  observed  that  the  sensitivity  of  the  various  tests 
for  nonrandomness  of  observations  depends  upon  the  type  of  non¬ 
randomness  which  is  present,  In  cases  where  this  can  be  anti¬ 
cipated  from  technical  considerations,  the  appropriate  test 
may  be  selected  in  advance,  but,  if  the  choice  of  tests  is 
based  on  an  examination  of  the  results,  the  significance  levels 
are  clearly  biased.  An  obvious  alternative,  that  of  subjecting 
all  data  to  the  same  series  of  tests,  is  not  free  from  criticism 
since  many  of  the  tests  are  not  independent,  although  the  degree 
of  dependence  is  not  yet  known,  It  would  appear  that  without 
technical  guidance,  the  best  procedure  is  probably  to  use  a 
series  of  tests,  interpreting  the  significance  levels  as  a  general 
Indication  rather  than  a  specific  prediction. 

We  wish  to  thank  John  Price  of  Hershey  Foods  who  in  the  early  stages  pro¬ 
grammed  for  us  the  Kendall  S  with  the  assistance  of  Brent  Burkholder.  This 
was  later  taken  over  by  Emerson  Evelhock  who  expanded  the  work  to  useful 


4° 


graphs.  The  demand  forecast  and  trend  simulation  model  vras  developed  by  CDR 
Tom  Bunker,  SC,  USN.  Throughout  eil  this,  Charla  Schaeffer  typed  end  retyped 
end  steyed  the  course. 
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DEMAND  OBSERVATIONS 
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KENDALL'S  S  STATISTIC 
WINDOW  -  4 
HDTR-4 


PERIOD 

DEM  OBS 

KENDALL 

'S  S 

PROBABILITY 

1 

6 

2 

10 

3 

8 

4 

10 

3 

0.271 

5 

8 

-2 

0.375  NO  TREND 

6 

10 

2 

0.375 

7 

5 

-3 

0.271 

8 

7 

-2 

0.375 

9 

9 

. 

0 

0.625 

10 

8 

- 

4 

0.167 

11 

11 

- 

4 

0.167 

12 

13 

- 

4 

0.167 

13 

10 

- 

2 

0.375  TREND  (1) 

14 

17 

- 

2 

0.375 

15 

19 

- 

4 

0.167 

16 

18 

- 

4 

0.167 

17 

21 

- 

4 

0.167 

18 

30 

_ 

4 

0.167 

19 

27 

- 

4 

0.167 

20 

39 

- 

4 

0.167 

21 

42 

- 

4 

0.167 

22 

43 

* 

6 

0.042  TREND  (2) 

23 

51 

* 

6 

0.042 

24 

55 

* 

6 

0.042 

25 

57 

* 

6 

0.042 

26 

57 

_ 

5 

0.1045 

27 

53 

-1 

0.5 

28 

57 

-1 

0.5 

29 

56 

-1 

0.5 

30 

59 

4 

0.167  NO  TREND 

31 

54 

-2 

0.375 

32 

56 

-1 

0.5 

33 

58 

0 

0.625 

34 

52 

_ 

0 

0.625 

35 

52 

- 

-3 

0.271 

36 

47 

* 

-5 

0.1045 

37 

44 

* 

-5 

0.1045 

38 

44 

* 

-5 

0.1045  TREND  (3) 

39 

39 

* 

-5 

0.1045 

40 

39 

- 

-4 

0.167 

41 

27 

* 

-5 

0.1045 

42 

31 

-3 

0.271 

43 

35 

0 

0.625 

44 

28 

2 

0.375 

45 

31 

-1 

0.5 

46 

29 

-2 

0.375 

47 

34 

4 

0.167  NO  TREND 

48 

28 

-2 

0.375 

49 

26 

on 

-4 
_  0 

0.167 

CRITICAL  VALUE  S-5 


IV 

ACTUALITY 


T 

NT 

T 

9 

1 

NT 

16 

21 

25 

22 

36% 

95% 
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KENDALL'S  S  STATISTIC 
WINDOW  -  6 
HDTR-4 


PERIOD 

DEM  OBS 

KENDALL'S 

S 

PROBABILITY 

1 

6 

2 

10 

3 

8 

4 

10 

5 

8 

NO  TREND 

6 

10 

5 

0.235 

7 

5 

-5 

0.235 

8 

7 

-5 

0.235 

9 

9 

-4 

0.2675 

10 

8 

- 

0 

0.559 

11 

11 

- 

5 

0.235 

12 

13 

* 

13 

0.0083 

13 

10 

★ 

9 

0.068  TREND  (1) 

14 

17 

* 

9 

0.068 

15 

19 

* 

11 

0.028 

16 

18 

* 

9 

0.068 

17 

21 

* 

11 

0.028 

18 

30 

* 

13 

0.0083 

19 

27 

* 

11 

0.028 

20 

39 

* 

11 

0.028 

21 

42 

* 

13 

0.0083 

22 

43 

* 

13 

0.0083  TREND  (2) 

23 

51 

* 

13 

0.0083 

24 

55 

* 

15 

0.0014 

25 

57 

* 

15 

0.0014 

26 

57 

_ 

14 

0.00485 

27 

53 

8 

0.102 

28 

57 

6 

0.1855 

29 

56 

0 

0.559 

30 

59 

2 

0.4  NO  TREND 

31 

54 

0 

0.559 

32 

56 

2 

0.4 

33 

58 

0 

0.559 

34 

52 

_ 

-4 

0.2675 

35 

52 

- 

-8 

0.102 

36 

47 

* 

-8 

0.102 

37 

44 

★ 

-12 

0.01815 

38 

44 

* 

-13 

0.0083  TREND  (3) 

39 

39 

* 

-13 

0.0083 

40 

39 

* 

-13 

0.0083 

41 

27 

★ 

-13 

0.0083 

42 

31 

_ 

-11 

0.028 

43 

35 

-8 

0.102 

44 

28 

-6 

0.1855 

45 

31 

-2 

0.4 

46 

29 

2 

0.4 

47 

34 

0 

0.559  NO  TREND 

48 

28 

-4 

0.2675 

49 

26 

oo 

-4 
_  c 

0.2675 

n  i  occ 

CRITICAL  VALUE  S-9 


IV 

ACTUALITY 


T 

NT 

T 

20 

2 

NT 

5 

18 

25 

20 

80% 

90% 

54 


PERIOD  DEM  OBS  KENDALL'S 

1  6 

2  10 

3  8 

A  10 

5  8 

6  10 

7  5 

8  7 

"irpaffg  f  ]  wit  nm-T - 

KENDALL'S  S  STATISTIC 

WINDOW  -  8 

HDTR-4 

S  PROBABILITY 

NO  TREND 

-4  0.36 

9 

9 

_ 

-8 

0.199 

10 

8 

- 

-4 

0.36 

11 

11 

- 

2 

0.452 

12 

13 

★ 

13 

0.0715 

13 

10 

* 

13 

0.0715 

14 

17 

* 

22 

0.0028 

TREND  (1) 

15 

19 

★ 

22 

0.0028 

16 

18 

* 

20 

0.0071 

17 

21 

* 

22 

0.0028 

18 

30 

* 

22 

0.0028 

19 

27 

* 

22 

0.0028 

20 

39 

* 

24 

0.00087 

21 

42 

* 

24 

0.00087 

22 

43 

* 

24 

0.00087 

TREND  (2) 

23 

51 

* 

26 

0.00019 

24 

55 

* 

26 

0.00019 

25 

57 

* 

26 

0.00019 

26 

57 

_ 

27 

0.0001075 

27 

53 

- 

21 

0.00495 

28 

57 

- 

19 

0.01155 

29 

56 

- 

13 

0.0715 

30 

59 

- 

13 

0.0715 

NO  TREND 

31 

54 

1 

0.5 

32 

56 

-4 

0.36 

33 

58 

4 

0.36 

34 

52 

-1 

0.5 

35 

52 

- 

-12 

0.089 

36 

47 

★ 

-16 

0.031 

37 

44 

* 

-21 

0.00495 

38 

44 

* 

-20 

0.0071 

TREND  (3) 

39 

39 

* 

-24 

0.00087 

, 

40 

39 

* 

-25 

0.00053 

41 

27 

* 

-25 

0.00053 

42 

31 

-24 

0.00087 

43 

35 

- 

-20 

0.0071 

44 

28 

- 

-18 

0.016 

45 

31 

- 

-14 

0.054 

46 

29 

-10 

0.138 

NO  TREND 

47 

34 

-1 

0.5 

48 

28 

2 

0.452 

49 

26 

-12 

0.089 

50 

29 

-10 

0.138 

CRITICAL  VALUE  S-13 


IV 

ACTUALITY 


T 

NT 

T 

20 

9 

NT 

5 

9 

25 

18 

80% 

50% 

55 


KENDALL'S  S  STATISTIC 
WINDOW  -  10 
HDTR-4 


PERIOD 

DEM  OBS 

KENDALL'S 

S 

PROBABILITY 

1 

6 

2 

10 

3 

8 

4 

10 

5 

8 

NO  TREND 

6 

10 

7 

5 

8 

7 

9 

9 

10 

8 

- 

-3 

0.431 

11 

11 

- 

-1 

0.5 

12 

13 

- 

13 

0.146 

13 

10 

- 

13 

0.146 

14 

17 

* 

25 

0.014  TREND  (1) 

15 

19 

* 

30 

0.00345 

16 

18 

* 

37 

0.00018 

17 

21 

* 

37 

0.00018 

18 

30 

* 

37 

0.00018 

19 

27 

* 

37 

0.00018 

20 

39 

★ 

37 

0.00018 

21 

42 

* 

39 

0.000058 

22 

43 

* 

41 

0.000015  TREND  (2) 

23 

51 

* 

41 

0.000015 

24 

55 

* 

41 

0.000015 

25 

57 

* 

43 

0.0000028 

26 

57 

42 

0.0000089 

27 

53 

- 

36 

0.000325 

28 

57 

- 

36 

0.000325 

29 

56 

- 

30 

0.00345 

30 

59 

- 

30 

0.00345  NO  TREND 

31 

54 

- 

18 

0.066 

32 

56 

9 

0.242 

33 

58 

7 

0.3 

34 

52 

_ 

-7 

0.3 

35 

52 

- 

-12 

0.168 

36 

47 

* 

-17 

0.078 

37 

44 

* 

-29 

0.0046 

38 

44 

* 

-32 

0.0017  TREND  (3) 

39 

39 

* 

-37 

0.00018 

40 

39 

* 

-36 

0.000325 

41 

27 

* 

-40 

0.0000365 

42 

31 

• 

-40 

0.0000365 

43 

35 

- 

-36 

0.000325 

44 

28 

- 

-35 

0 . 00047 

45 

31 

- 

-30 

0.00345 

46 

29 

- 

-26 

0 . 01115  NO  TREND 

47 

34 

-17 

0.078 

48 

28 

-14 

0.127 

49 

26 

-15 

0.108 

50 

29 

-  6 

0.332 

CRITICAL  VALUE  S-18 


IV 


ACTUALITY 

T  NT 

T 

17 

13 

NT 

7 

6 

24 

17 

71% 

35% 

56 


KENDALL'S  S  STATISTIC 
WINDOW  -  12 
HDTR-4 


PERIOD  DEM  OBS  KENDALL'S  S  PROBABILITY  CRITICAL  VALUE  S-22 
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Abrams  Tank  Inspect  And  Repair  Only  As  Necessary  (IRON) 

Economic  Model 

1.  Introduction 

On  22  March  1969  Lieutenant  General  Jimmy  D.  Ross  the  Deputy 
Chief  of  Staff  for  Logistics  proposed  an  initiative  to  establish 
a  point  in  the  life  of  an  Abrams  tank  when  a  depot  level  inspeat 
and  repair  only  as  necessary  (IRON)  program  may  be  appropriate  to 
extend  the  service  life  of  the  tank.  He  recommended  that  TACOM 
develop  an  Economic  Model  to  provide  the  cost-benefit  of  this 
IRON  program.  This  model  should  predict  the  point  in  time  when  a 
tank  becomes  an  IRON  candidate.  As  proof  of  the  benefit,  the 
model  must  predict  the  extension  of  service  life  and  the  reduc¬ 
tion  of  operating  and  sustainment  costs  through  this  remaining 
life,  finally,  the  results  should  be  validated  in  a  hardware 
demonstration. 

In  response  to  this  request  TACOM  and  PM  Abrams  have  devel¬ 
oped  the  Reliability  Centered  IRON  (RCIRON)  program.  (The  RCIRON 
program  differs  from  the  IRON  program  in  that  only  certain 
candidate  components  are  inspected  and  repaired.)  Tor  the  purpose 
of  a  validation  test  60  M1IP  tanks  were  transferred  from  Germany 
to  the  National  Training  center  (NTC),  Fort  Irwin,  California. 
Fourteen  of  these  tanks  were  selected  for  the  RCIRON  demonstra¬ 
tion.  These  tanks  were  sent  to  Anniston  in  December  1989  where 
the  RCIRON  inspection  was  performed.  These  60  tanks  are  currently 
being  tested  at  the  NTC.  The  goal  of  the  RCIRON  program  is 
"preventive  correction  of  impending  failure"  which  can 

Decrease  the  maintenance  burden  on  field  units. 

Extend  "combat  life” 

Decrease  field  Operating  and  Support  (O&S)  costs. 

Our  problem  is  to  develop  an  optimum  maintenance  policy  for 
the  implementation  of  the  RCIRON  for  the  entire  M1/M1A1  fleet. 
This  requires  the  establishment  of  periodic  inspections  for  a 
system  composed  of  many  parts,  each  of  which  has  its  own  failure 
rate  and  its  own  required  frequency  of  repair.  Since  each  item 
has  its  own  expected  time  between  failures,  each  will  have  its 
own  optimum  time  for  periodic  inspection. 

To  maintain  a  system  at  the  desired  level  of  operation  a 
regular  routine  of  test  and  inspection  must  be  established.  This 
type  of  inspection  is  normally  performed  at  the  organisational 
level.  The  maintenance  organisation  may  also  perform  preventive 
periodic  inspections  and  repairs  on  a  scheduled  list  of  items  or 
subsystems.  There  are,  however,  some  parts  of  the  system  which 
may  require  inspection  and  repair  at  a  higher  echelon,  such  as  a 
depot.  It  is  assumed  that  by  scheduling  the  inspection  of  item  a 
little  before  the  expected  failure,  the  number  of  failures  and 
the  costs  of  unscheduled  downtime  can  be  reduced  enough  to  offset 
the  cost  of  scheduled  maintenance.  This  is  the  basis  of  the 
RCIRON  program.  Fifty  parts  have  been  identified  as  candidates 
for  the  RCIRON  Program.  If  there  is  evidence  of  failure  or 
deterioration  at  the  time  of  inspection  these  parts  will  be 
repaired  or  replaced. 
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2.0 


A  large  number  of  the  models  reviewed  during  this  study 
develop  the  optimum  replacement  time  for  a  single  type  of  part  or 
component.  A  model  proposed  by  Samuel  B.  Richmond  was  suggested 
as  an  economic  useful  life  model  for  the  RCIRON  program. 

The  model  equation  is 
were 

t 

tf<t)dt+Tf(t)-p.JL 
,J0  A** 


T  ■  the  preventive  maintenance  period 
p  ■  probability  of  failure  before  time  T 
f(t)  ■  cumulative  probability  of  failure 
A  ■  cost  of  overhaul  after  failure 
B  ■  cost  overhaul  before  failure 

This  model  considers  that  if  the  cost  a  scheduled  replacement  is 
less  than  the  cost  of  a  field  replacement  then  the  scheduled 
replacement  is  cost-effective.  The  model  assumes  that  an  item 
will  always  be  replaced  at  RCIRON  inspection,  and  the  main  cost 
will  be  the  downtime  required  to  perform  the  maintenance.  The 
nature  of  the  RCIRON  process  is  to  inspect  a  component  and 
replace  it  only  if  necessary.  This  type  of  model  does  not  meet 
the  requirements  of  the  RCIRON  program. 

3.0  Dmfc.axlQg.aA.ing.. Harkov. 1-r.acsss  Hods  1 

A  more  realistic  model  is  presented  by  A.  S.  Goldman  and  T. 
B.  Slattery  in  their  1964  book  Maintainability.  With  some  modifi¬ 
cation  this  model  can  be  adapted  for  the  RC-IRON  program.  This 
model  assumes  that  periodio  inspections  will  occur  at  scheduled 
periods  of  time,  miles  or  rounds.  At  the  tine  of  these  periodio 
inspections  there  is  a  probability  that  the  item  will  havo 
deteriorated  and  should  be  replaced.  There  also  exists  a  proba¬ 
bility  that  the  item  will  have  failed  prior  to  the  scheduled 
inspection.  Associated  with  these  inspections  and  replacements 
are  incremental  costs  which  contribute  to  the  total  maintenance 
costs . 

3.1  MQjflmL.Beauix.mmt.nti 

Tl\e  goal  of  a  maintenance  model  is  to  develop  an  optimum 
maintenance  policy.  In  order  to  develop  a  realistic  model  and  to 
optimise  the  system  retain  certain  system 
characteristics  must  be  determined. 

These  characteristics  are, 

For  each  item 

1.  Deterioration  probability. 

2.  Probability  of  failure  during  deteriorated  state. 

3.  Failure  probability  during  normal  state. 

4.  Average  hours  for  preventive  inspection. 

5.  Average  hours  for  preventive  repair. 

6.  Average  elapsed  time  for  field  diagnosis  and  repair 
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7.  Average  hours  for  field  ropsir 
For  tho  system! 

1.  Cost  of  sotting  up  for  provontivo  inspection. 

2.  Depot  costs  per  hour. 

3.  Field  costs  per  hour. 

4.  Cost  of  downtime 

When  this  information  is  available  the  eatheeatical  model 
can  be  developed.  The  objective  is  to  develop  a  method  of 
applying  mathematics  and  economic  theory  to  preventive 
maintenance  problems.  The  method  described  applies  only  to 
preventive  maintenance  situation  where  the  failure  pattern 
ie  predictable.  Thie  pattern  ie  generally  aseociated  with 
wear  out  failure  distributions. 

3.2  MmllummtiA.fl.ml  .Modal 

The  periodic  inspection  model  can  be  represented  by  the 
following  expression! 

qjs d&q&m. 


Wherei 

T  ■  Scheduled  inspection  period.  (Time,  miles, 
rounds ) 

i  ■  ith  item  to  be  inspected. 

Cl  ■  Unit  aost  of  maintenance  and  repair  of  the 
ith  item  of  the  system. 

Cl  1  ■  Cost  of  periodic  inspection  of  the  ith  item. 

C21  ■  Cost  of  preventive  repair  of  the  ith  item. 

C31  ■  Cost  of  field  failure  of  the  ith  item. 

PI  ■  Probability  of  repair  of  ith  item. 

El  *  Expected  number  of  failures  between  periodic 
inspections 

The  Goldman/Slatterly  (G/S)  model  considers  that  the 
main  cost  of  maintenance  is  tho  cowl  ol  loul.  downtime.  Thin 
is  also  the  main  cost  driver  in  the  AMSAA  model.  The  AMSAA 
model  also  includes  the  cost  of  the  average  hours  required 
to  replace  the  item.  The  G/S  model  also  includes  hours,  but 
in  a  different  way.  The  G/S  model  considers  hours  more  as 
an  overhead  factor.  For  the  PC-IRON  program  the  AMSAA 
method  will  be  used. 
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The  coat  of  inspection  and  repair  ean  than  ba  written  aai 

cijac^trij+cgrijMnrij 

ca4»cydM,+ai&*nu4 


Where i 

CU  ■  Coat  of  periodic  inapaotion  of  tha  ith  itea. 
C21  ■  Coat  of  pravantiva  rapair  of  tha  ith  item. 

C31  ■  Coat  of  flald  failura  of  tha  ith  itea. 

CD  ■  Coat  of  downtiae.  ($1781/day). 

Ill  ■  Doantiaa  dua  to  inapaotion. 

CIU  ■  Coat/hour  of  inapaetora. 

Till  «  Houra  required  for  inapaotion. 

D21  ■  Doantiaa  dua  to  dapot  rapair. 

CR21  *  Coat/hour  of  dapot  rapair. ( $46. 69/Hr). 

TR21  ■  Houra  required  for  dapot  rapair. 

D31  ■  Doantiaa  dua  to  fiald  rapair. 

CR31  ■  Coat/hour  of  flald  repair. ( $104/Hr) 

TR31  ■  Houra  required  for  fiald  rapair. 


3.3  Xathl.ayet  «»■-., Coi.t 


The  total  ayateas  coat  ia  aada  up  of  tha  ooat  of 
setting  up  tha  inapaotion  process,  tha  overhead  ooata  of 
aaintaining  the  inapaotion  and  rapair  facility  and  tha  aua 
of  all  tha  part  inapaotion  and  rapair  ooata.  If  tha 
inapaotion  and  rapair  faoility  was  dedicated  to  tha  HI 
prograa  tha  total  overhead  ooata  would  have  to  ba  included 
in  tha  ooat  of  preventive  inapaotion  and  rapair.  Tha  depota 
however  are  funded  aaparately  and  tha  faoility  ooat  can  be 
left  out  of  tha  ooat  equation.  The  expraaaion  for  total 
ayataa  ooat  oan  be  written  aa i 


Were  i 

C  •  Total  Syataa  Coat 

Co  ■  Coat  of  aatting  up  inapaotion 

Cl  ■  Inapaotion  and  repair  ooata  of  the  ith  itaa. 

T  ■  Scheduled  inapaotion  period.  (Time,  alias, 
rounda  ) 

3.4  Eailuji. . and  .Deterioration  f r.ahabillt.y, 

In  order  for  the  RC-IRON  prograa  to  be  auccaaaful 
iteaa  nuat  ba  replaced  before  they  have  failed.  One  of  tha 
goala  of  tha  RC-IRON  prograa  ia  to  develop  cathode  of 
deteoting  the  degree  of  deterioration  of  the  iteaa 
inspected.  When  thia  haa  been  done  an  itea  will  be  in  one 
of  three  atatea  at  the  end  of  a  fciae  interval.  The  action 
taken  will  depend  on  whio^  state  the  itea  is  in  at  tha  tiae 
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of  inapaotion.  Tha  atata  and  aotiona  takan  ara  ahoxn 
in  Table  I. 

Tahla  I 

Sinta  ...ftl.Itti  Action  Taken 

1.  Good  Nona. 

2.  Datarioratad  Rapair  at  next  aohadulad 

inapaotion. 

3.  Palled  Rapair  innediately. 

Tha  data  ourrantly  availabla  from  Saapla  Data  Collaotion 
( 8DC )  provldaa  an  eetiaate  of  tha  probability  that  an  ltaa 
haa  aithar  failad  or  la  In  good  condition.  Thla  oan  ba 
axpraaaad  aat 

R+0-1 


Wharai 


Jt  •  axp  (-axe*) 


and 

alpha  and  B  ■  paranatara  of  Weibull  dlatrlbutlon. 
t  ■  tine,  ailaa  or  rounda. 

R  ■  Rallablllty. 

0  ■  Probability  of  a  failura. 

A  nodal  of  tha  RC1R0N  prooaaa  nuat  Inoluda  a  third 
probability  and  tha  atata  of  tha  ltaa  oan  ba  axpraaaad  aai 

p  1  +  p  2  +  p  J  ■  1 


Wharai 

p  1  *  probability  that  itan  ia  in  good  condition, 
p  2  ■  probability  that  itan  ia  in  datarioratad  atata. 
p3 ■  probability  that  itan  ia  in  a  failad  atata. 

Tha  nathod  uaad  to  aolva  for  tha  failura  and  deterioration 
probabilitiaa  in  tha  0/3  nodal  la  a  deteriorating  Markov 
prooaaa.  Tor  a  given  interval  of  tine  t  each  itan  can  ba 
oharactarisad  by  tha  probabilitiaa  pi.)  that  it  atarta  in 
tha  tine  interval  in  atata  i  and  anda  in  atata  j .  Aa  an 
axanpla  pi#  1  ia  tha  probability  that  tha  itan  ranaina  in  tha 
good  condition  through  tha  interval  and  pi. 3  ia  tha 
probability  that  itan  faila  during  the  tine  period.  Thaaa 
probabilitiaa  oan  ba  arranged  in  a  natrix  aa  a  apaoial  fora 
of  a  Markov  prooaaa.  Thia  natrix  ia  known  aa  the  tranaition 
natrix . 
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Arranged  in  a  matrix  thasa  probabil itiaa  arai 


maaxnnu) 


anuLiiAMi) 


A,.  *.» 

*k.i 

e  o 


AT 


Tha  procaaa  which  can  ba  daaoribad  in  thia  manner  is  a 
apaoial  eaaa  of  a  Markov  ohain.  Tha  probabilities  that  an 
item  Mill  be  in  the  ith  state  at  tha  and  of  the  n+lth  time 
interval  are  given  by 


jpil(a)  , pi  (a)  ipf  (a) 


*.•  JV.d 

0  A,  I  PtJ 


pi(a*l)  ,pl(a+l) 


10  0 


Mhioh  can  be  abbreviated  as 

[PiU)J(pij)  -  [PKn+U] 

Since  the  item  is  assumed  to  be  good  at  the  beginning  the 
condition  for  the  item  at  the  start  oan  be  written  as 

[PitOI]  ■  [1,0.0] 

The  condition  of  the  item  after  n  transitions  can  then  be 
written  as 

[H(n)]  -  [Pi(0)]  ( pi)n 


If  T  is  the  period  batwaan  preventive  inspeotiona  and  v  is 
tha  pariod  of  transition  tha  numbar  of  pariods  batwaan 
inspaotions  is  agual  to 

H  ■  T  /  t 


Tha  numbar  of  pravantiva  repairs  during  tha  life  of  the 
tank  would  than  ba  equal  to 


Mara 


Tha  expected 
expression 


H  ■  pi!  x  T1  /  T 

Pi  ■  numbar  of  pravantiva  repairs 
p  12  ■  probability  of  transition  to  tha 
deterioration  state. 

T1  ■  Useful  life< time, miles, rounds ) 

T  ■  Pariod  batwaan  inspeotiona 
numbar  of  failures  can  ba  computed  using 


tha 


*r 


WJ) 


67 


4.  Canoimi 

A  as  jor  ir«i  of  eohoorn  is  tho  applicability  of  the  deteri¬ 
orating  Harkov  to  a  aystaa  were  tha  tiaa  between 
inspaotiona  is  larga.  Host  applioations  of  this  aodal 
raquira  that  tha  tisa  t  should  ba  short/ enough  so  that  only 
ona  ohanga  of  state  is  likely  but  should  ba  suffieiantly 
long  so  that  a  repair  eould  ba  dona  in  i  ties*  Another 
problas  is  tha  use  of  a  constant  transition  matrix 
regardless  of  tha  age  of  tha  system*  These  faotors  do  not 
affect  tha  validity  of  tha  basio  modal  but  limit  tha 
usefulness  of  tha  methods  of  calculating  tha  failure  and 
deterioration  probabilities. 

Tha  modal  as  presented  represents  a  simplified  solution  to 
tha  preventive  maintenanoe  problem  and  presents  the  basics 
involved.  There  is  very  little  data  available  on  the 
deterioration  rate  of  HI  components.  It  is  hoped  that  the 
results  of  the  RCXRON  will  provide  some  clues  as  to  whloh 
components  exhibit  deterioration.  Without  this  data  the 
model  given  is  of  little  practical  value.  The  example, 
however,  serves  to  illustrate  the  elements  involved  in  a 
more  general  treatment  of  the  RC-XRON  problem. 

An  extensive  literature  search  failed  to  turn  up  an  analyt¬ 
ical  model  that  satisfies  all  the  requirements  of  the 
RC-XRON  process.  To  over  oome  this  and  to  develop  an  idea 
of  how  the  system  would  preform  it  was  decided  to  develop  a 
simulation  model. 


B.  Simulation . Hod  ml. 

The  data  required  by  a  simulation  model  is  the  same  as  that 
used  in  the  Harkov  process  model .  The  input  requirements 
for  the  RCXRON  model  listed  in  paragraph  2.2  will  provide 
the  information  necessary  for  a  simulation  model. 

Tor  each  item  the  following  parameters  are  required i 

1.  Hiles  to  deterioration  ■  Tt 

2.  Hiles  to  failure  during  deteriorated  state  ■  Di 

3.  Hiles  to  failure  during  normal  state  »  Ri 

4.  Average  hours  for  preventive  inspection  ■  DTX1 
B.  Average  hours  for  preventive  repair  >  RTD1 

6.  Average  down  time  at  depot  ■  DTDi 

7.  Average  down  time  for  field  diagnosis  and 
repair  ■  DTF1 

9.  Average  hours  for  field  repair  RTF1 

10.  Hiles  at  next  failure  ■  NTj,l 

11.  Hiles  at  next  deterioration  ■  HR j,l 

12.  Cost  of  preventive  inspection  ■  INSPCi 

Tor  the  system! 

1.  Cost  of  setting  up  for  preventive  Inspection  "XNSUP 
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2.  Depot  repair  costs  por  hour  ■  $46.69 

3.  Field  repair  ooets  per  hour  ■  $104.00 

4.  Coat  of  doMntiee  ■  $1704.00 

5.  Znspeotion  sohedule  (miles)  ■  INSPECT 

6.  Hilee  at  next  inspection  ■  NEXTINSP 

7.  Total  vehiole  mileage  ■  TMILES 

8.  Vehicle  life  ■  20  years. 

9.  Average  annual  aileage  ■  TMXLES 

10.  LXfE  -  20*YMILES 

A  component  Mill  be  in  either  of  three  stateei 

1.  Operational 

2.  Deteriorated 

3.  Tailed 

If  a  component  fails  it  is  repaired  and 
replaced  immediately  and  returned  to  the  operational  state. 
Xf  a  component  is  in  the  deteriorated  state  it  remains  in 
that  state  until  the  next  scheduled  inspection.  Each 
component  can  remain  in  the  deteriorated  state  for  a 
limited  number  of  miles.  Xf  the  deterioration 
miles  plus  the  deterioration  period  is  less  than  the  miles 
for  the  next  inspection  the  part  fails  and  is  repaired  or 
replaced.  The  simulation  begins  by  calculating  the  miles  to 
the  next  failure  and  the  next  change  to  the  deterioration 
state.  This  is  represented  byi 
NT1.1  ■  Ti 
NRM  -  Ri 

The  times  to  failure  are  computed  using  a  Weibull  distribu¬ 
tion.  The  expression  for  this  calculation  is  given  ast 

1  "  EXP (  LOG (  LOG (  l/(  l-RND)  )  ) -LOG (  ALPHA! ) /BETA! ) 

where 

ALPHA  is  the  soale  parameter 
BETA  is  the  shape  parameter 

RND  is  a  random  uniform  deviate 

The  time  to  deterioration  and  the  duration  of  the 
deteriorated  state  are  not  well  known.  Little  or  no  data 
has  been  collected  showing  these  factors.  In  order  to  r  u  n 
the  model  assumptions  have  been  made  to  generate  the 
deterioration  time  and  duration  of  the  deteriorated  state. 
The  first  assumption  is  that  the  time  for  a  component  to 
enter  the  deteriorated  utate  is  less  than  the  time  to 
failure.  Components  are  likely  to  deteriorate  before  they 
fail.  The  time  spent  in  the  deteriorated  state  would  depend 
on  a  variety  of  maintenance  factors.  The  deterioration 
could  be  so  subtle  that  the  tank  crew  may  not  realise  that 
a  problem  exists.  Other  deterioration  would  be  so  obvious 
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that  the  oraw  would  repair  or  replace  tho  component  after  a 
vary  short  datarioration  period.  Tor  tha  saka  of 
demonstrating  tha  modal  it  la  aaauaad  that  tha 
datarioration  distribution  ia  alao  a  Weibull  diatribution 
with  a  aoala  paraaatar  2  tiaaa  greater  than  tha  ALPHA  of 
tha  failura  diatribution.  Tha  expression  for  tha  tiaa  to 
datarioration  iat 


T}>axp( log( log( 


1 

1  -rnd 


) ) )  -log( 


f 


Tha  lifa  of  tha  tank  ia  aaauaad  to  ba  twanty  yaara  and  tha 
avaraga  yaarly  allaaga  ia  about  1000  ailaa.  It  la  alao 
aaauaad  that  a  datarioratad  component  would  not  go 
undataotad  for  6  aontha  or  500  ailaa.  Tha  tiaa  for  a 
ooaponant  to  raaain  in  tha  datarioratad  atata  ia  tha 
dlffaranoa  batwaan  tha  failura  tiaa  and  tha  datarioration 
tiaa. 

Tha  rulaa  for  failing  or  passing  lnspaetlon  arat 

1.  If  MTli  and  NRli  <  NEXTINSP  than  tha  ooaponant  will  fail 
and  TMILE8  will  ba  agual  to  aithar  NTl#i  or  NR, i  whioh  avar 
la  tha  saallast  and  a  naw  NT1  #  i  and  a  naw  NRl,i  will  ba 
oalaulatad  aai 


NTS# i  -  TNILE8  +  T1 
NR2# i  -  THILE8  ♦  Ri 

2.  If  NTl#i  >  NEXTIN8P  and  NR1,1  <  NEXTIN8P  than  tha 

ooaponant  will  fail  at  tha  aohadulad  lnspaetlon  and  b  a 
rapairad  or  raplaoad.  THILES  will  ba  oat  agual  to  NIXTINSP 
and  a  naw  NTJ#X  and  a  naw  NRj#i  will  ba  oaloulatad  aai 

NTS# i  -  THILE5  +  T1 
NR2# i  -  THILES  +  Ri 
THILES  ■  NEXTINSP 

Than  a  naw  NEXTINSP  will  ba  calculated  aai 

NEXTINSP  -  NEXTINSP  ♦  INSPECT 

3.  If  NT1# i  <  NEXTINSP  and  NR1 # i  >  NEXTINSP  than  tha 
ooaponant  will  fail  and  ba  raplaoad  iaaadiataly.  THILES 
will  ba  sat  agual  to  NTl#i  and  a  naw  NTj.i  and  a  naw  NRj#i 
will  ba  oaloulatad  aat 

NT2# 1  -  THILES  +  Ti 
NR2# i  -  THILES  +  Ri 
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If  NT2,i  >  NEXTINSP  them 


TMXLES  -  NEXTINSF 

And  4  now  NEXTINSP  Mill  be  calculated  as  i 

NEXTINSF  ■  NEXTINSF  ♦  XN8FECT 

At  each  step  in  the  o4loulation  the  cost  associated  Mith 
the  inspection*  fitld  and  dapot  rapalr  ara  aoouaulatad 
using  tha  expressions! 

rCOSTi  -  rCOSTi  *  DTfi  +$1784  +  RFTi  *  $104 

RCOST1  -  RCOSTi  *  DTDi  +$1784  ♦  RTDi  *  $104 

ICOST1  *  ICOSTi  +XNSPC1 

Tha  itaration  continues  until  TKILES  >■  LITE.  Aftar  aaoh 
run  tha  total  ooat  of  aaintananoa  for  tha  vahiola  lifa  la 
oaloulatad  and  printad  in  a  fila  for  furthar  analyaia.  Tha 
aohadulad  ailaa  batwaan  inapaotiona  la  inoraaantad  and  tha 
alaulation  la  rapaatad  until  tha  aohadulad  aaintananoa 
pariod  ia  equal  to  tha  lifa  of  tha  vahiola. 

ft.  Cong  1 us Ians 

Preliminary  runa  of  tha  nodal  vara  aada  uaing  varioua 
datarioration  factor*  and  inopeotion  tinea  from  1000 
ailaa  to  20000  ailaa.  Tha  raaulta  obtained  fron 
running  tha  aodel  Mara  diaappointing •  Tha  nodal  did 
not  daaonatrata  that  there  Maa  any  advantage  to  tha 
RCIRON  procaaa.  Sinoa  va  Mara  dealing  Mith  an 
unproven  nodal  uaing  fiotitioua  data  Me  decided  to 
Malt  until  tha  data  fron  Anniston  Dapot  and  tha  taata 
at  tha  NTC  vara  available  before  presenting  the 
raeulta  obtained  by  running  tha  nodal. 

Tha  data  obtained  fron  Anniaton  did  not  provide  ua  Mith 
any  additional  aatinataa  of  datarioration  tinea. 

Tha  data  fron  tha  taata  at  NTC  hoMaver  Mere  very 
revealing.  The  taata  at  NTC  were  performed  by  three 
different  organitatlona .  Each  organisation  waa  aaaigned  to 
NTC  for  about  tMo  Meeke  during  Mhioh  tine  it  waa  planned 
that  the  tanka  would  travel  about  300  ailaa.  Each  of  theaa 
period*  is  called  a  rotation.  During  tha  first  rotation  th* 
raw  data  showed  that  RCIRON  tank*  had  a  mean  miles  between 
maintenance  actions  (MMBMA)  of  almost  twice  that  of  the 
control  tanks. 

Thar*  were  several  sources  of  variation  affecting  th* 
results  of  th*  testa.  Th*  primary  source  of  variation  was 
th*  differences  between  th*  RCIRON  and  th*  control  tanks. 
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Tha  other  known  aouroaa  art  tha  diffaronoo  batwaan 
rotationa  and  tha  faot  that  aaoh  tank  waa  drivan  a 
diffarant  nuabar  of  ailaa.  In  ordar  to  datarnina  tha  dagroa 
of  variation  batwaan  tha  RCIRON  tanka  and  tha  oontrol  tanka 
tha  othar  aouroaa  of  variation  auat  ba  raaovad. 

Savaral  atatiatioal  taat  wara  parfornad  to  datarnina  tha 
dagraa  of  tha  RCIRON  tanka  and  tha  oontrol  tanka.  Tha 
diffaranoa  batwaan  tha  two  aanpla  during  rotation  #1  if 
vary  aignifioant.  Tha  diffaranoa  during  tha  othar  two 
rotationa  ia  nuoh  laaa  aignifioant. 

During  rotation  #3  tha  taat  ahow  that  wa  oan  ba  nora 
than  90%  confidant  that  tha  two  aanpla  ara  tha  aana.  Tron 
thia  wa  oan  oonoluda  that  tha  banafit  of  tha  RCIRON  prooaaa 
ia  diaaipatad  within  tha  firat  1000  nilaa  aftar  oonplation. 
Thaaa  raaulta  ara  ainilar  to  thoaa  of  tha  ainulation  nodal. 


< 
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A  Model  for  Optimally 
Reducing  Uncertainty 
Andrew  Anderson  Thompson  III 
Ballistic  Research  Laboratory 


Abstract 

This  paper  presents  a  model  that  can  be  used  for  the  optimal  reduc¬ 
tion  of  uncertainty.  There  are  three  major  components  of  the  model.  An 
a  priori  surface  used  to  describe  the  original  state  of  the  world  represents 
the  measure  that  is  to  be  minimized.  A  function  representing  the  reduc¬ 
tion  of  the  measure  as  a  function  of  the  resource  needs  to  be  supplied. 
Based  on  these  two  functions  a  surface  describing  the  optimal  allocation 
of  the  resource  can  be  derived. 


Introduction 

This  paper  presents  a  model  that  can  be  used  for  the  optimal  reduc¬ 
tion  of  uncertainty.  The  components  of  the  model  will  be  presented  with 
some  examples  of  their  interactions  and  then  several  successful  interpre¬ 
tations  of  the  model  will  be  discussed.  The  paper  concludes  by  suggesting 
some  possible  interpretations  of  the  model.  Hopefully,  the  reader  will  be 
able  to  find  some  useful  interpretations  of  this  model. 

There  are  three  major  components  of  the  model.  An  a  priori  surface 
used  to  describe  the  original  state  of  the  world  represents  the  measure 
that  is  to  be  minimized.  A  function  representing  the  reduction  of  the 
measure  as  a  function  of  the  resource  needs  to  be  supplied.  Based  on 
these  two  functions  a  surface  describing  the  optimal  allocation  of  the 
resource  can  be  derived. 

Uncertainty  Surface 

The  uncertainty  function  represents  the  distribution  of  the  attribute 
that  is  to  be  minimized.  In  the  case  of  a  lifeboat,  a  bivariate  normal  distri¬ 
bution  with  a  variance  dependent  on  time  provides  a  useful  realization  of 
the  uncertainty  function  as  the  probable  unknown  location.  In  many  cases 
the  uncertainty  will  be  based  on  odds  and  thus  will  be  a  likelihood  surface. 

Information  Function 

The  information  function  reduces  the  uncertainty  as  a  function  of  the 
resource  applied.  This  is  the  mechanism  that  transfers  the  measure  of  the 
uncertainty  function  to  a  discrete  category  representing  gain.  This  gain 
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can  be  thought  of  as  knowledge.  A  desirable  property  to  require  is  that 
the  remaining  uncertainty  at  a  point  is  the  same  for  two  applications  of 
resource  to  a  point  and  for  an  amount  of  resource  equal  to  the  sum  of  the 
two  but  only  applied  once.  Mathematically,  the  following  property  is 
required 


f(l(p)/l+r2)mf(f(l(p)/l)/2). 


In  the  above  equation,  l(p)  represents  the  likelihood  or  uncertainty  at 
a  point  and  r  represents  an  amount  of  resource.  A  function  that  has  this 
property  is  the  exponential  function.  Using  the  exponential  function  as  the 
information  function  and  w(p)  to  represent  the  amount  of  reduction  at  a 
given  point  the  amount  of  uncertainty  remaining  can  be  represented  as 

The  amount  transferred  to  expected  information  is 


Optimal  Resource  Surface 

Koopman  (1979)  presents  a  theorem,  based  on  methods  used  by  J. 
Willard  Gibbs  in  thermodynamics,  to  define  the  properties  of  a  likelihood 
surface  after  an  optimal  resource  allocation  is  completed.  After  an 
optimal  allocation,  the  likelihood  density  will  have  equal  values  in  the  area 
of  allocation  and  be  less  in  the  other  areas.  Performing  an  optimal  alloca¬ 
tion  amounts  to  passing  a  plane,  that  is  parallel  to  the  independent  vari¬ 
ables  through  the  target  likelihood  density  so  that  the  mass  above  the 
plane  is  equal  to  the  available  resource.  An  optimal  allocation  divides  the 
potential  area  into  an  area  of  resource  application  and  an  area  to  be 
ignored.  In  the  area  of  allocation,  the  effort  expended  at  each  location  will 
be  proportional  to  the  amount  of  likelihood  mass  that  is  above  the  "cutoff' 
plane.  One  way  to  visualize  this  is  to  consider  the  change  in  the  gain  of 
information,  and  always  apply  resource  to  the  regions  associated  with  the 
largest  gains.  First  consider 

— /(p)(l-e'w(p)r)|ri,0«  l(p)w(p) 

dr 

Imagine  a  gridlike  partition  of  l(p)w(p ) . 
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Allocate  resource  to  the  cells  with  the  largest  value  until  they  hit  the  level 
of  the  next  lower  level. 


Repeat  this  until  all  the  resource  is  used. 

The  result  is  a  surface  that  is  flat  in  the  region  of  uncertainty  reduction. 
One  can  imagine  that  the  grid  becomes  the  set  of  rational  numbers. 

Example  1. 

As  an  illustration  of  the  above  ideas,  three  methods  for  solving  a  single 
problem  will  be  discussed.  Consider  a  3x3  matrix,  where  the  value  in  cell 
(ij)  [i  and  j  running  from  1  to  3]  represents  the  uncertainty  associated 
with  cell  (i,j). 


i-1 

0.20 

0.10 

0.05 

-2 

0.10 

0.30 

0.05 

*3 

0.05 

0.10 

0.05 

j-1 

*2 

-3 

The  probability  associated  with  the  region  corresponding  to  cell  (2,2) 
is  0.3.  Assume  there  is  a  fifty-percent  reduction  in  the  probability  if  a  unit 
of  resource  is  applied  to  a  cell.  Fifty  percent  is  transfered  to  the  catagory 
information  gain.  What  sort  of  resource  allocation  will  maximize  the  pro¬ 
bability  for  twenty  units  of  resource? 

1.  Method  One:  Proceed  sequentially  using  a  maximum- likelihood 
method;  apply  each  unit  to  the  area  with  the  highest  probability.  After 
each  unit  is  applied,  replace  the  original  cell  [call  it  C]  with  "C  (1-DF),"  or, 
here,  (0.3  x  1/2  *)  0.15.  (That  is,  some  of  the  probability  is  moved  to  the 
category  of  knowledge.)  For  the  allocation  of  the  twenty  rounds,  the  pro¬ 
cedure  would  follow  this  pattern: 


Step  1:  Choose  cell  (2,2),  since  it  contains  the  greatest  probability. 
Replace  0.3  with  (0.5  x  0.3  * )  0.15. 


Step  2:  Choose  cell  (1,1),  since  it  now  contains  the  greatest 
probability. 

Replace  0.2  with  (0.5  x  0.2  = )  0.1. 
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Step  3:  Choose  cell  (2,2),  since  it  now  contains  the  greatest 
probability. 

Replace  0.15  with  (0.5  x  0.15  =)  0.075 

Now  several  cells  have  the  equal  maximum  probability  of  0.10.  Here, 
one  can  randomly  choose  any  one  of  those  cells.  (When  this  possibility 
exists ,  one  car  only  speak  of  "an"  optimal  solution.) 

Method  One  not  only  indicates  the  number  of  units  to  be  applied  to 
each  cell  but  also  indicates  the  best  sequence  for  delivery.  This  straight¬ 
forward  method  can  be  used  when  one  has  a  discrete  uncertainty  surface. 

2.  Method  Two.  This  method  introduces  the  techniques  used  when  one 
has  a  continuous  uncertainty  surface.  The  method  is  to  find  the  amount 
of  resource  to  apply  to  the  highest-valued  cell  to  reduce  it  to  the  level  of 
the  cell  containing  the  next  lower  value.  Then  apply  the  resource  to  both 
of  those  cells  until  level  of  the  next  lower  cell  is  reached.  In  this  situation, 
we  assume  that  the  effort  is  also  continuous  (i.e.,  that  the  resource  can  be 
applied  in  fractional  amounts).  This  assumption  is  necessary  in  order  to 
calculate  the  optimal-resource  surface. 

If  a  unit  is  applied  to  cell  (2,2),  then  the  probability  remaining  would 
be  0.15.  Since  this  value  is  lower  than  0.2,  the  reduction  has  gone  too  far. 
The  proper  amount  to  apply  to  cell  (2,2)  is  a  fractional  amount,  enough  to 
have  reduced  it  to  0.2  and  no  lower.  Thus: 

Step  1:  Reduce  the  value  of  cell  (2,2)  to  the  value  of  cell  (1,1) 

0.2  *0.3*0.5"n-*n-  0.585. 

Step  2:  Find  the  amount  of  effort  that  will  reduce  0.2  to  0.1 

0.1  »  0.2*0.5**n-*n  *  1 

Step  3:  Find  the  amount  of  effort  that  will  reduce  0.1  to  0.05 

0.05  *  0.1*0.5**n-+  n  »  1 

The  derivation  thus  far  has  not  made  use  of  the  fact  that  the  resource 
is  limited.  In  this  continuous  case,  the  optimal  allocation  of  ammunition 
can  now  be  derived  by  means  of  an  "Effort  Matrix"  expressed  in  terms  of 
"E"  -  the  "minimal-effort  value."  The  value  in  each  cell  represents  the 
effort  or  resource  to  be  applied  to  the  corresponding  area. 

For  the  Effort  Matrix  corresponding  to  the  given  problem,  each  of 
the  ceils  with  probabilities  of  0.05  has  some  undetermined  amount  of 
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effort  E  in  it.  The  cells  corresponding  to  the  probabilty  ceils  with  entries 
of  0.1  have  one  full  unit  more  of  effort  in  them,  so  they  carry  the  value 
1 +E.  The  cell  corresponding  to  the  value  of  0.2  has  yet  one  more  unit  of 
effort,  so  its  entiy  is  2+E.  Finally,  the  cell  with  a  value  of  0.3  has  an  addi¬ 
tional  0.58  units  of  effort  (rounded  to  two  decimal  places).  Its  entry  is 
2.58+E. 


2  +  E  1  +  E  E 

1  +  E  2.58  +  E  E 

E  1  +  E  E 

The  total  effort  expended  is  to  be  equal  to  twenty  so  by  summing  the 
cells  of  the  matrix  and  setting  the  total  equal  to  twenty,  we  can  find  the 
optimal  effort  for  each  cell: 


20  -  9E  +  7.58 
E  -  1.38 

Thus,  where  the  resource  can  be  applied  continuously,  the  optimal  solu¬ 
tion  is: 


3.38 

2.38 

1.38 

2.38 

3.96 

1.38 

1.38 

2.38 

1.38 

Returning  to  the  original  case  of  20  integral  units,  if  the  resource 
could  not  be  subdivided,  it  would  be  necessary  to  enter  integer  numbers 
of  rounds  in  each  cell.  As  an  approximation,  we  would  round  off  the 
values  in  the  cells  to  integer  amounts  and  make  further  adjustments  to 
ensure  that  the  sum  is  twenty.  One  plausible  solution  in  this  case  is 

3  3  1 

3  4  1 

1  3  1 

3.  Method  Three.  This  method  uses  the  ideas  of  Koopman;  Gibbs  (1928) 
originally  applied  these  ideas  to  a  physics  problem.  Koopman  derives  two 
formulas  that  can  be  used  to  find  the  optimal  resource  surface.  The  first 
formula  is  used  to  find  the  area  to  apply  resource.  After  this  area  has 
been  defined,  the  second  formula  is  used  to  determine  the  amount  to 
apply  to  each  point.  This  method  is  valid  when  the  TLD  is  continuous 
and  is  a  formalization  of  the  technique  used  in  Method  2.  The  equations 
used  are: 

*  =  //  [(In  (P(x,y)  w(x,y))  -  In  A)/w(x,y)]  dxdy  (1) 

A 
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and 


^iy) 


In 


(2) 


9  is  the  total  effort  4>  vx,y)  is  the  resource  density  at  x,y 

p(x,y)  is  the  probability  at  x,y 
A  is  the  height  of  the  cut  off  plane 


A  is  the  area  of  application 

e-w(*>v)  js  residual  probability  located  at  (x,y). 

Equation  1  divides  the  uncertainty  region  into  the  two  areas  based  on 
the  amount  of  resource  available.  Equation  2  is  used  to  determine  the 
allocation  at  each  point.  As  applied  to  the  current  example  the  steps  are 
as  follows: 

First,  express  the  residual  probability  as  an  exponential.  As  we  apply 
more  resource  to  a  specific  region  the  returns  on  each  unit  diminish  in 
proportion  to  the  probability  that  remains  in  that  region.  This  diminishing 
rate  of  return  is  captured  by  the  exponential  function. 

Second,  solve  Equation  1  for  A.  A  is  the  height  of  the  plane  that  cuts  the 
uncertainty  surface  at  the  level  appropriate  for  that  amount  of  effort. 
(Note  that  integration  can  be  replaced  by  summation  for  this  discrete 
case). 

Third,  find  the  amount  of  effort  at  each  of  the  nine  points  using  Equation 

2. 

Fourth,  find  an  integer  solution. 

Implementing  these  steps  yields  the  following: 

.  Pt  ->w(xy)  --In.  5  -  .6931 

Note  that  w(x,y)  is  constant  and  can  be  replaced  by  w. 

9 

20  «  j]  (In  (p.  *  w)  -  In  A )/w 
«-i 

20w  -  2  In  p.  w 

In  A- - —  A  =  .0133 

-9 
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Using  <j>.  =  —  In 
w 


we  get  the  optimal  effort  matrix 


3,3819  2.3818  1.3817 

2.3818  3.9669  1.3817 

1.3817  2.3818  1.3817 


Integerization  o'  this  solution  yields  the  same  result  as  the  previous  exam¬ 
ple.  Note  that  this  method  will  work  for  a  continuous  uncertainty  surface. 

Next  an  example  using  continuous  resource  on  the  bivarite  normal  distri¬ 
bution  is  presented.  It  is  hoped  that  this  example  will  serve  to  clarify  all 
the  above  ideas. 

Bivariate  Normal 

In  the  continuous  case  Koopman’s  result  guides  us  in  finding  the  value  for 
the  cutoff  plane  for  a  given  amount  of  effort.  In  using  this  equation  note 
that  the  resource  surface  must  be  greater  than  zero  at  all  points;  we  can¬ 
not  use  negative  amounts  of  resource  at  an  unlikely  area  and  counterbal¬ 
ance  this  by  applying  more  resource  to  more  probable  area.  The  value  z 
■  In  (A)  is  the  cutoff  plane  of  the  surface  In  (p(x,y)  w(x,y)).  All  regions 
where  In  (p(x,y)  w)  <  In  (A)  are  ignored.  Assuming  w(x,y)  *  w  and  In  (A) 
can  be  expressed  as  In  (p  (x  'y ')  w)  Equation  1  can  be  expressed  as 

$  88  //[in  (p(x,y)  w)  -  In  (p(x  ',y ')  w)]dxdy/w.  (3) 

A 

This  can  be  rewritten  as 

w  $  =  //  [in  (p(x,y))  +  In  w  -  ln(p(x  ',y '))  -  In  wjdxdy  (4) 
A 

s  //  (In p(x,y)  -  In  p(x  ',y  '))dxdy 
A 

In  this  situation  Equation  4  shows  In  A  is  inversely  related  to  w.  Equation 
2  can  be  written  as 

<Kxf)  ^  —  (In  p{xy)  -  In  p(x  >  ')!•  (5) 

w 

This  indicates  the  amount  of  resource  applied  to  each  point  is  propor¬ 
tional  to  the  difference  between  the  distribution  and  the  cutoff  plane. 
These  ideas  are  applied  to  the  following  problem. 

Example  2: 
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Given  a  circular  normal  probability  surface  and  constant  information 
function,  describe  the  optimal  resource  surface.  The  uncertainty  is 
described  by 

•(«V) 

p(/cy)  “  (2nai)'1e  lr  . 

Taking  the  natural  log  we  have 

2 .1  -C«2+y2) 

ln/?(jc,y)  ■  \n(2n<r)  + - 

to1. 


Rewriting  Equation  1  for  this  problem  we  have 


9  i  -(x2+v2) 

In  (2  ir  ay  + - -  +  In  w  -  In  A 

2cr2 


•-// 

A 

The  integrand  is  the  equation  of  a  concave  downward  parabaloid. 


dxdy/w. 


Next  change  to  polar  coordinates 

2kA  „  2 

A4  * 


wQ  -  JJ 
00  L 


ln(2iro-)“  + - +  lnw-lnA 

2cr2 


RdRd  a 


(6) 


The  value  in  brackets  as  previously  mentioned  must  be  greater  or  equal  to 
zero.  In  terms  of  A  we  will  use  the  following  expression  for  In  A 

-A2 

In  A  -  In  (2  7T  a2)'1  + - +  In  w. 


2  <r 


Equation  6  can  be  written 
2k  A  _ 


W  $  ■  // 


-R 

In  (2  n  er2)"1  + - +  In  w  -  In  (2  ?r  cr2)*1 


00  L 

2k  A  „  3 

-R3 


-// 


0  0 
2*, 
f 


/ 

o  v 


1 2er 
4 


A2R 1 
2<r2  J 


2<r 

dRd© 


jll 

2  o'2 


-  lnw 


RdRd© 


,44' 

- +  — 

o  2  A  2 

8  a  4a 


d© 


80 


2  ir  A4 


i tA4  a  4  w  <fHr2 

- -+A  = - 

4  (j“  «■ 


A  «  (2  7T  cr2)’1  e 


i  1/2 


7T 


The  optimal  resource  surface  will  be  zero  outside  the  circle  of  radius  A; 
within  the  circle  the  allocation  is  given  by  Equation  2  which  simplifies  to 

The  information  gain  for  this  example  can  be  found  by  the  following 
method.  Note  from  Figure  1  that  P(information)  is  the  volume  bound  by 
the  curve  p(R,0)  and  the  cutoff  plane  z  «*  A  or  the  difference  between 
Figure  la  and  lb.  The  volume  under  the  circular  normal  distribution  is 
given  by 


This  volume  contains  a  cylinder  of  radius  A  and  height  p(A);  so  we  must 
remove  this  volume  from  the  previous  value.  The  volume  of  the  cylinder 
is 

1/2  -1 ,  .1/2 

1  l  ' 

- e 

2na2 


nA2h 


2ircr 


vw 


n 


so  the  expression  for  p(information)  is 


1  .  , 

!  +  “(—) 
<r  n 


1/2 


9  * 

e 


(7) 


Next  we  extend  this  problem  to  quantify  the  relationship  between 
intelligence  or  reduced  variance  of  the  uncertainty  distribution  and 
increasing  the  amount  of  resource.  Assume  a  circular  normal  density  with 
sigma  of  100.  Let  w  *  .5  (the  probability  of  missing  the  target  is  e'  5  or 
.61)  and  suppose  there  are  ten  units  of  resource,  each  unit  having  an 
effective  radius  of  thirty  meters.  Notice  that  both  the  standard  deviation 
of  the  uncertainty  and  the  total  amount  of  effort  need  to  be  in  the  same 
units;  thus  the  total  effort  needs  to  be  an  area.  For  this  example  the  total 


81 


effort  available  ($)  is  the  number  of  resource  units  multiplied  by  the  area 
each  covers  or  10  *  k  *  302.  From  Equation  7  above  the  probability  of  hit 
is 


1  + 


1  ( 10*.5  *900*^1 


1/2 


100 


7T 


100 


-(4500) 


i/J 


■  .146 


If  we  instead  use  20  units,  the  information  gain  is  ,246;  however,  if  we  had 
reduced  the  target  location  error  by  fifty  percent  so  sigma  was  50  for  10 
units  the  p(information)  would  be  .388.  A  reduction  in  sigma  to  70 
increases  the  probability  of  information  to  the  same  level  as  doubling  the 
amount  of  resource.  These  observations  give  guidelines  for  analysis  of  the 
benefits  of  intelligence  versus  increasing  the  resource  allocation. 


Theory  of  Search 

Koopman  worked  for  the  Navy  during  WWII  and  directed  his  energies 
toward  solving  problems  associated  with  getting  convoys  across  the  Atlan¬ 
tic,  finding  submarines,  and  directing  maritime  rescue  operations.  During 
the  course  of  his  work  he  developed  a  formal  theory  of  search.  The  model 
presented  by  this  paper  is  a  slight  generalization  of  search  theory.  In 
order  to  describe  the  a  priori  location  density  of  a  target,  knowledge  of 
the  situation  based  on  past  experience  or  functional  relationships  must 
suffice.  For  maritime  rescue  operations,  the  bivarite  normal  distribution 
proved  to  be  an  adequate  model.  The  detection  function  was  based  on 
models  of  human  observation  from  airplanes  and  several  adequate 
glimpse  models  were  derived.  Using  these  functions  it  was  posible  to  allo¬ 
cate  search  time  judisously.  There  was  a  major  problem  getting  convoys 
across  the  Atlantic.  Using  destroyer  escort  time  as  a  resource  to  minimize 
the  probability  of  attack  by  a  submarine  and  reasoning  about  submarine 
tactics  to  build  up  probability  of  attack  surfaces;  it  was  possible  for  Koop¬ 
man  and  his  associates  to  make  reasonable  allocations  of  destroyer  time 
and  substantially  decrease  the  sucessful  the  effects  of  the  submarines. 


Artillery  Effectiveness 

Problems  associated  with  increasing  the  effectiveness  of  indirect  artillery 
fire  have  been  addressed  by  Sandmeyer(1986).  Targeting  errors,  target 
size,  and  errors  associated  with  a  round  need  to  be  considered. 

In  order  to  see  the  connection  between  theory  of  search  problems  and 
artillery  problems,  think  of  an  artillery  round  as  the  detection  function. 
T  he  round  searches  for  the  target.  For  a  given  number  of  rounds,  the  goal 
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is  to  diminish  the  target  likelihood  density  as  much  as  possible.  The 
expected  damage  of  a  round  is  the  convolution  of  the  precision  errors  and 
the  damage  function  of  the  round.  For  artillery  systems  this  convolution 
will  usually  involve  the  bivariate  normal  distribution  and  the  von  Neuman 
•  Carlton  damage  function.  The  target  likelihood  density  is  the  convolu¬ 
tion  of  the  target  area,  the  target  location  errors,  and  the  artillery  mean 
point  of  impact  errors.  The  latter  two  are  both  assumed  to  be  bivariate 
normal  distributions,  while  the  target  area  function  is  usually  a  rectangle 
whose  height  is  the  reciprocal  of  the  target  area.  The  optimal  amount  of 
target  damage  can  be  found  by  using  these  two  convoluted  functions  and 
Koopman’s  theorem. 

Our  total  effort  is  packaged  into  a  discrete  number  of  artillery 
rounds,  or  expected  damage  functions,  each  of  which  have  the  same  area 
effects.  It  is  not  possible  to  apply  the  indicated  optimal  effort  or  damage 
to  each  point;  thus,  it  is  impossible  to  achieve  the  optimal  amount  of  dam¬ 
age.  This  optimal  damage  surface  will  serve  as  an  upper  bound  on  perfor¬ 
mance. 

Recalling  that  the  target  likelihood  density  is  broken  into  a  part  to  be 
searched  and  an  area  to  be  ignored,  several  theoretical  observations  can 
be  made.  First,  the  amount  of  damage  occurring  outside  the  optimal  area 
is  wasted  effort  in  the  sense  that  it  could  be  applied  more  effectively  else¬ 
where.  Next,  damage  exceeding  that  indicated  by  the  cutoff  plane  in  the 
search  area  is  wasted  effort.  The  best  approach  is  to  try  to  approximate 
the  optimal  effort  surface  by  the  summation  of  translated  expected  dam¬ 
age  functions.  In  most  cases  this  will  result  in  a  target  likelihood  density 
with  ripples  near  the  level  of  the  optimal  cutoff  plane. 

Richard  Sandmeyer  (1986)  of  the  AMSAA  developed  a  technique  to 
optimally  approximate  the  optimal  effort  surface.  There  are  three  major 
sections  to  his  method. 

1.  Calculate  the  upper  bound  and  optimal  effort  surface. 

2.  Calculate  an  approximate  solution  by  using  either  integer  program¬ 
ming  or  least  squares  techniques  on  a  restricted  domain.  The  res¬ 
tricted  domain  consists  of  all  the  points  defined  by  the  intersections  of 
grid  that  includes  the  optimal  effort  surface. 

3.  Starting  with  the  previous  solution,  use  a  steepest  descent  technique 
to  arrive  at  the  solution. 

The  theoretic  upper  bound  is  found  based  on  Koopman’s  theorem. 
The  optimal  effort  surface  can  be  calculated  functionally  for  each  point. 
In  finding  this  surface,  it  is  assumed  there  are  no  restrictions  on  the  effort 
applied  at  each  point. 
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Step  two  of  Sandmeyer’s  method  involves  finding  a  close  approxima¬ 
tion  of  the  best  set  of  translated  damage  functions.  This  is  accomplished 
by  looking  at  a  discrete  number  of  points  and  optimizing  the  approxima¬ 
tion  over  this  set. 

The  third  step  of  his  method  starts  with  the  result  of  step  two  and 
allows  the  aimpoints  to  drift  into  the  positions  of  the  best  approximation. 
Computationally  the  third  step  is  the  most  expensive.  Happily,  for  most 
situations  the  result  of  the  second  step  is  within  a  few  percent  of  the  upper 
bound.  This  method  requires  too  much  CPU  time  to  be  used  in  the  field. 
As  Sandmeyer  has  already  suggested  the  performance  of  this  method  can 
be  used  to  evaluate  the  effectiveness  of  other  methods.  It  is  clear  that  the 
approach  developed  by  Sandmeyer  is  not  limited  to  solving  problems 
associated  with  artillery. 

Medical  Diagnosis 

In  this  situtation,  an  unknown  desease  presents  itself  through  a  subject  or 
patient  as  a  set  of  symptoms.  The  goal  is  to  identify  the  desease  or  short 
of  that  find  a  treatment  that  gets  rid  of  the  symptoms.  The  parameter 
space  is  the  set  of  all  known  medical  ailments;  the  goal  is  to  reduce  the 
uncertainty  as  to  the  patient’s  condition  in  some  optimal  fashion.  The 
resource  available  to  the  physician  is  money,  which  is  translated  into 
information  via  various  microbiological  and  chemical  tests  and  through 
response  to  particular  medical  treatments.  An  a  priori  likelihood  surface 
can  be  built  up  from  existing  desease  frequencies,  knowledge  of  the 
patient,  and  risk  factors.  After  this  a  sequence  of  treatment  and  testing 
can  be  developed  that  would  be  optimal  in  reducing  the  uncertainty  sur¬ 
face  associated  with  the  patient.  This  would  give  both  the  patient  and  the 
physician  a  formal  method  that  explains  a  particular  course  of  action. 

Funding  Problems 

This  class  of  problems  arise  when  there  is  a  limited  amount  of  resource  to 
spend  on  a  number  of  problems.  The  uncertainty  surface  can  be  inter¬ 
preted  as  the  likelihood  of  benefit  to  society.  The  different  programs  will 
have  different  reduction  rates  and  these  can  be  estimated  from  historical 
data  (or  from  pilot  studies).  In  this  case  funding  would  allocated  automat¬ 
ically  after  the  politicians  characterized  the  uncertainty  surface  and  the 
social  scientists  evaluated  the  effectiveness  of  the  various  programs. 

Conclusion 

It  is  my  hope  that  the  ideas  developed  by  search  theory  will  find  a  wider 
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application  when  presented  as  a  general  model  for  reasoning.  Sand- 
meyers  work  has  increased  the  domain  of  the  problems  by  developing 
approximation  methods.  On  an  intuitive  level  this  model  of  reasoning  can 
be  seen  to  operate  in  many  fields.  The  advantages  of  having  a  formal 
model  are  to  be  seen  in  a  wider  scope  of  applications  and  in  more  effi¬ 
cient  problem  solving. 
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Abstract 

This  paper  describes  a  method  to  detect  and  segment  ground 
targets  in  IR  imaging  systems  using  statistical  models  for  target 
and  background.  The  detection  method  is  an  iterative  technique 
that  subtracts  the  background  from  the  scene,  using  a  double  window 
filter  with  the  assumed  independent  distribution  function  for 
background.  The  outer  size  of  the  double  window  is  determined 
iteratively  by  the  largest  connected  component  remaining  in  the 
scene.  The  segmentation  or  boundary  detection  methods  employs  the 
well-known  statistical  theory  of  change  detection  in  speech 
segmentation.  The  likelihood  ratio  test  is  modified  to  improve  the 
performance.  The  design  method  was  initially  applied  on  laser 
radar  range  imagery,  but  simulation  results  showed  that  the  method 
works  well  for  both  flir  and  laser  radar  range  image,  hence 
promising  an  advanced  technique  for  multisensor  fusion  algorithm. 
Hypothesis  modelling  for  target  and  background  is  analyzed  and  a 
brief  discussion  of  the  fusion  algorithm  is  also  included. 

i.  -  introduction 

The  double  window  filtering  technique  has  been  used  by  many 
investigators  for  target  detection  for  several  years  [1].  The  double 
window  filter  approach  is  a  general  technique  which  tests  a 
statistic  calculated  from  the  target  and  the  background.  The  sizes 
of  the  inner  and  the  outer  windows  of  the  filter  are  usually  fixed, 
according  to  an  estimation  of  the  smallest  and  the  largest  target 
sizes.  Several  factors  limit  the  performance  of  the  filter,  such 
as  the  requirement  of  knowledge  of  target  size  ,  time  consuming  and 
the  erratic  behavior  of  the  pixel  classification  near  the  target 
boundary  or  inside  a  bimodal  target.  In  this  paper,  we  propose  a 
technique  that  attacks  all  those  above  problems  by  :  i)  updating 
apriori  knowledge  of  target  size  by  a  feedback  loop,  ii)  speed  up 
the  computational  time  of  the  filter  by  jumping  the  window  instead 
of  sliding  it  pixel  by  pixel,  and  iii)  testing  the  statistic  of  a 
pixel  by  generating  several  outer  window  statistics  instead  of 
using  its  neighboring  pixel.  The  detection  method  is  further 
improved  by  incorporating  a  second  test,  which  measures  the 
statistic  of  a  connected  component  against  its  immediate 
surrounding  pixels  to  reject  possible  clutter  components.  The 
detected  boundary  is  then  refined  using  the  log-likelihood  ratio 

1 


87 


Preceding  Page  Blank 


teat  which  detects  the  boundary  without  requiring  a  threshold. 
This  likelihood  ratio  test  is  modified  with  apriori  knowledge 

(i.e.,  knowing  the  direction  of  changes)  to  improve  the 

performance. 

2.  -  Hypothesis  modelling  for  target  and  background 

Theorem  [2]  s  let  xl,x2, . . xn  be  independent  and 

identically  distributed  (iid)  random  variables 

and  Sn  -  xl  +  x2  + . +  xn 

Sn  -  E  (  Sn  ) 

let  Y  -  - — - 

CK  (  Sn  ) 

where  E(Sn)  is  the  estimation  of  Sn  and  o*  (Sn)  is  the 
standard  deviation  of  Sn, 
then  random  variable  Y 

a)  has  mean  0,  variance  of  1 

b)  is  the  sum  of  small  independent  random  variables. 

2.1  -  Double  window  filter 

The  double  window  filter  provides  a  technique  to  test  the 
statistics  of  the  pixels  inside  the  inner  window  against  the  pixels 
inside  the  outer  window,  see  Figure  1. 


rzooM 


XMMBft  WISDOM (X  .If) 


Xij  -  E  {  xij  such  that  ijeo.w  } 

Let  Sij  ■  ———————— - — -  for  i j  4  I . W 

o*  {  xij  such  that  ijao.w  ) 

where  Xij  are  random  variables  of  the  array  or  image. 

If  the  pixels  inside  the  inner  and  the  outer  window  are  iid  then, 
Sij  has  mean  0  and  variance  of  l. 
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2.2  -  Hypothesis  models 


Let  Ho  denote  the  hypothesis  for  background 
HI  denote  the  hypothesis  for  target 
and 

Hypothesis  Ho  :  {  Rlj  /  B  -  ®#  } 

versus  yt  e, 

Hi  :  {  Rlj  /  9  -  e,  } 


Here  Rlj  represents  the  Intensity  of  the  ljth  pixel  and  eij 
represents  a  feature  extracted  from  Rlj  that  we  may  use  to 
distinguish  the  target  from  the  background.  In  flir  imagery,  the 
thermal  intensity  of  the  target  is  usually  different  from  the 
surrounding  background.  In  laser  radar  range  imagery,  at  low 
depression  angle,  the  target  oan  be  distinguished  from  the 
background  by  observing  the  variation  of  the  measured  ambiguous 
range,  because  it  is  often  reasonable  to  assume  the  background 
consist  of  rough  surfaces.  Therefore,  a  3X3  window  is  used  to 
convert  the  range  image  to  the  variance  image  as  a  preprocessing 
(or  feature  extraction)  step  before  using  the  double  window  filter. 

Hence, 

for  each  pixel  inside  the  inner  window,  compute 


Sij 


o.w 


eij  -  e  {  eij  such  that  ij  «  o.w  } 
o*  {  eij  such  that  ij  *  o.w  } 
outer  window 


where  eij  is  the  mean  of  Rij  for  flir  images  and  eij  is 
the  3x3  window  variance  of  Rij  for  laser  radar  range  images.. 

Then, 

Ho  :  Sij  -  N  (  0  ,  1  ) 

HI  :  Sij  not  belong  to  Ho 

where  N(0,l)  denotes  a  normal  distribution  with  mean  0  and 
variance  of  1.  Note  that  the  approximated  normal  distribution  is 
reasonable  for  both  sensors  because t  i)  for  flir  imagery,  the 
thermal  intensities  can  be  assumed  to  be  small  independent  random 
variables;  ii)  for  laser  radar  range  imagery,  the  3X3  window 
variances  of  the  ambiguous  ranges  were  approximated  to  be 
multivariate  Gaussian  variables[3] . 

3.  -  Implementation  of  the  detection  algorithm 

Figure  2  illustrates  the  implementation  of  the  feedback  loop 
for  an  ROI  (region-of-interest)  detection  algorithm  with  two 
options  labeled  as  #1  and  #2.  The  first  option  determines  the  size 
of  the  largest  connected  component  in  the  scene,  whereas  the  second 
option  determines  the  largest  sizes,  in  horizontal  and  vertical, 
of  the  connected  components  remaining  in  the  scene  after  performing 
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FIGURE  2  •  BLOCK  DIAGRAM  OF  THE  DETECTION  METHOD 


a  simple  clutter  raj action  taat.  Tbo  outar  window  aiza  is  first 
initial izad  and  than  ths  doubla  window  moves  across  tha  antira 
imaga  in  such  a  way  that  its  innar  windows,  whioh  its  size  is  half 
of  tha  outar  window  siza,  do  not  overlap,  but  ara  adjacent  to  each 
other.  This  speeds  up  the  computational  time  of  the  doubla  window 
algorithm,  compared  to  the  conventional  method  whioh  movas  pixel 
by  pixel [i].  Each  pixel  inside  tha  innar  window  is  tested  against 
tha  statistics  of  pixels  inside  the  outar  window,  for  which  the 
connected  components  are  formed  by  pixels  passing  a  threshold  value 
which  is  normalized  to  the  outer  window  statistics  (explained  in 
section  3.1  below).  The  sizes  of  these  connected  components  ara 
than  determined  and  fad  back  to  the  double  window  filter  process. 
This  process  is  iteratively  repeated  while  accumulating  the 
statistic  for  each  pixel  for  each  iteration.  Pixel  classification 
is  then  performed  to  form  ROI  connected  components.  Each  ROI  is  now 
tested  against  its  immediate  surrounding  background  pixel  statistic 
to  reject  possible  clutter  component. 

3.1  -  Hypothesis  test  #1  (  pixel  classification  ) 

Recall  that  Sij  is  assumed  to  have  mean  0  and  variance  of  1 
approximately  if  Sij  belonged  to  the  background.  Therefore,  a 
threshold  can  be  chosen  base  on  the  justification  of  the 
probability  of  error.  That  is,  the  probabilistic  value  of  Sij  can 
be  estimated  corresponding  to  ths  total  number  of  iterations,  and 
is  tested  to  determine  whether  or  not  Sij  belongs  to  the 
background,  which  has  mean  0  and  variance  of  1. 

For  example,  a  Student-t-test[4]  may  be  formed  as 

E  (Sij)  -  z« 

test  - - - -  <  t#  (Student-t-test) 

O'(Sij)/  \[rT 


where , 
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z*  :  critical  value  of  z  for  areas  under  the  normal  curve 
t„  :  critical  value  of  the  t-distribution 
n  :  total  number  of  iterations 
«  :  degree  of  confidence  level 

The  number  of  iterations  is  usually  greater  than  10  when  using 
the  student-t-test.  Moreover,  this  requires  the  accumulation  of  ali 
the  statistical  values  of  Sij. 

For  more  convenience,  fewer  computations  and  storages,  a 
binomial  trial  methc  l  can  be  used  to  test  the  hypothesis.  For 
example, 


let 

Xij  - 

0 

if  sij  <  z* 

" 

1 

otherwise 

and 

P(Xij 

- 

0)  -  P(Sij  <  z*  )  -  p 

then  for  n  trials,  the  probability  that  X  -  0  occurred  k  times  is, 

nl  k  n-k 

P(k)  - - p  (l-p) 

kl  (n-k) l 

Hypothesis  test  :  accept  HO  if  k  >  r 

Hi  if  k  <  r 

then 

P(H1/H0)  -  ^  F(k) 
k-0 

From  standard  statistical  tables  which  are  available  in  most 
statistical  text  books[4),  we  can  justify  n,  r,  and  p  to  obtain  the 
desired  value  of  P(H1/H0) .  For  example, 

From  Table  IV,  page  513  of  Reference^] ,  if  zM  -  0.9  then  p 

-  0.815  at  the  significance  level  of  0.0005,*  From  Table  II,  if  n 

-  5  and  r  -  1  then  P(H1/H0)  <  0.0067. 

Since  the  assumption  that  the  pixel  values  of  background  are 
independently  distributed  may  not  be  true  in  practice,  we  have  to 
justify  the  value  of  parameters  based  on  the  experiment.  The 
selocted  value  of  z«  can  be  determined  either  from  mathematic 
model  or  from  the  histogram. 

with  the  selected  value  of  zm  ,  connected  components  are 
formed  by  ijth  pixel  such  that  the  value  of  Sij  is  greater  than  z« 
for  each  iteration.  The  outer  window  size  can  be  determined 
according  to  the  sizes  of  these  connected  components  for  each 
iteration,  and  the  process  is  repeated  with  this  new  value. 

3.2  -  Hypothesis  Test  #2  (clutter  rejection) 

From  the  previous  process,  the  connected  componcntu  are  formed 
by  adjacent  pixels  such  that  for  n  iterations,  P(Xij  -  l)  >  l  - 
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P(H1/H0) .  In  this  process,  we  will  test  the  statistics  of  these 
components  against  the  immediate  surrounding  background  pixel*?  to 
reject  possible  clutter  components. 

Let  Ti  and  Bi  be  set  of  pixels  as  shown  in  Figure  3, 


BCt -  B(St) 


-  E  {  «ij  much  that  i j  e  z  ) 

-  9*  {  ei  j  such  that  i  j «  z  } 

The  Student <*t -test  may  be  used  in  this  oase.  Once  again, 
justification  must  be  based  on  the  real  data  experiments,  and  this 
test  (SNR  test)  works  as  a  simple  clutter  rejection  method. 
Suppose  we  have, 

Hypothesis  HO  :  D  <  l 

versus  HI  :  D  >  1 


and  let 

where 


E(«i  ) 
) 


then  P(H1/H0)  -  1  -  P<Z*  <  l)  -  1  -  0,8413  -  0.1587 

This  simple  clutter  rejection  test  uses  the  SNR  and  estimates 
the  sizes  and  shapes  of  the  connected  components  to  reject  possible 
alutter  components.  This  process  is  illutrated  in  Figure  2  as  the 
option  #2  in  the  feedback  loop. 

However,  the  final  clutter  rejection  test  is  more  important. 
This  test  is  derived  from  the  analysis  of  the  target  and  background 
statistics  to  enhance  the  performance. 

For  example,  let  D  Is  a  function  of  means  and  variances  of 
target  and  background, 

D  «  f(mT,o^T  ,mt,o^  } 

Assume  that  target  mean  and  variance  are  greater  than 
background  mean  and  variance,  respectively.  Then, 

6 


92 


D  -  (mv  o  <rr  )*(n»#  o  o\  ) 

where  o  denotes  a  proportional  operator  and  *  denotes  an 
improportional  operator, i.e, 


if  mr  >  mfc  then  nr  >  m|  >  0  or  mT/mf  >  l  . 

Therefore,  the  inproportional  operator  can  be  a  subtraction  or  a 
division. 

For  flir  imagery, 


assume 


then  D 


mT  >  nft  ,  o*r  +  o' 

yov*"'V  y  °t,+  «i 


For  laser  radar  imagery, 


assume 


then  D 


Hypothesis  HO  :  D  4  K 

HI  s  D  >  \ 

where  \  is  a  constant,  to  be  determined  from  the  empirical 
distribution.  Note  that,  this  constant  can  be  modeled  as  a  function 
of  range  and  weather  conditions.  In  fact,  these  factors  make  the 
target  distribution  function  varied  relatively  to  the  background 
distribution.  Therefore,  a  constant  setting  threshold  will  have 
false  alarms  or  miss  some  detections  in  some  images,  i.e.,  close 
or  long  ranges  images. 


4.  -  Boundary  .fittoatlon.  Mtthod 


4.1  -  Statistical  Theory  of  Change  Detection  [5] 

The  use  of  the  statistical  theory  of  change  detection  in 
speech  segmentation  has  been  known  and  very  successful.  The 
results  are  impressive  and  well  documented.  Since  the  methods  are 
general,  they  are  well  suited  for  application  to  the  detection  of 
the  boundaries  of  objeots  in  IR  images.  The  likelihood  ratio  test 
is  the  most  common  hypothesis  test  used  to  detect  the  time  change 
of  an  unknown  autoregressive  (AR)  model. 


Let 


p 

e  a,  y„  .  +  v„ 

i  m  l  1  " 
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where 


var(v„)  -  <r* 

®  -  (  a,  ,aa, . ,aa;  O'*  ) 

Hypothetic  HO  :  e  -  0,  1,2, . ,  n 

HI  :  6  -  e,  1,2, . r  r? 

9-0,  r+1, . . . . ,n 

r  and  9's  unknown 

Here  HO  denote  a  detection  of  no  change  and  Hi  denote  a 
detection  of  change. 

Likelihood  ratio  teat  follows 


max  max  min  log  L(H1/H0)  >  \ 

r  «,  •, 


which  yields, 


max  P(r)  >  A. 
r 


P(r)  -  n  log  o*9%  - 

r  log  of1  - 

1 

<r‘(W)  -  min  - 

1  (  y, 

e  (#w) 

k  €  w  s 

•  (W)  -  arg  min 

I 

1  <  yk  “  3 

•  k 

CM  *  i  « 

where  W  denotes  any  one  of  the 

P 

r 

i~i 


window  (1,2, . ,n}, 

(1,2, . ,r),  (r+l, . .’,n)  and  #  denotes  number  of  the  elements 

of  the  referred  set. 


4.2  -  Modified  likelihood  ratio  test 

For  our  application,  only  HI  is  considered  and  the  log 
likelihood  ratio  test  can  be  simplified  to, 

min  P(r)  ■  r  log  cr^r)  +  (n-r)  log  e^n-r) 

where,  <rl( r)  :  sampled  variance  of  the  set  (1,2,. ...,r) 

o*(n-r)  i  sampled  variance  of  the  set  (r+l, . . . . ,n) 

Appendix  A  gives  an  analytically  proof  of  the  log  likelihood 
ratio  test  base  on  the  "ideal-stochastic"  case.  From  the  analysis 
of  the  sampled  variances,  the  following  heuristic  tests  are 
developed  to  improve  the  performance. 
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Hypothesis  HO  :  9  -  N  (  mO  ,  0**0)  1,2, . ,  n 

Hi  :  e  -  N  (  ml  ,  <3**1)  1,2, . . r 

9  -  N  (  m2  ,  0**2)  r+l, . . n 

(  mi  ,  o'*!)  ars  sampled  maan  and  variance  of  the  referred  set. 
case  1  ;  ml  >  m2  ,  o'  l  f  o'  2 

min  P(r)  -  r(  2m0  -  ml  +  log  o' 1  )  +  (n-r)  (  m2  +  log  o' 2  ) 
case  2  :  ml  <  m2  ,  o'  1  ^  o'  2 

min  P(r)  -  r(  ml  +  log  o' 1  )  +  (n-r)  (  2mo  -  m2  +  log  0*2  ) 
case  3  :  ml  f  m2  ,  o'  l  >  O'  2 

min  P(r)  -  r(  20*  0  -  o'  l  +  log  ml  )  +  (n-r)  (  o' 2  +  log  m2  ) 
case  4  :  ml  ^  m2  ,  O'  1  <  o'  2 

min  P(r)  -  r(  0*1  +  log  ml  )  +  (n-r)  (  20' 0  -  a*  1  +  log  m2  ) 

Note  that  the  direction  of  changes  are  known.  The  improvement 
of  these  modified  tests  over  the  log  likelihood  ratio  test  was 
illustrated  in  the  previous  report[6)  by  mean  of  the  simulation. 

4.3  -  Implementation  of  the  Method 

As  previously  described,  the  detection  process  generates  a 
binary  ROI  image,  since  it  is  difficult  to  apply  the  test  in  two 
dimensional  data,  we  can  now  use  a  "spoke" [1],  as  shown  in  figure 
4  below,  collocated  with  each  ROI  to  obtain  sequences  of  data  in 
one  dimension  and  then  apply  the  hypothesis  test  above  to  detect 
the  boundary . 


noons  4 
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1)  For  each  line  of  the  spoke,  obtain  a  sequence  of  data  from 
the  original  image,  which  is  correspondingly  positioned  along  the 
line  and  where  its  length  is  twice  the  distance  from  the  centroid 
to  the  boundary  of  its  corresponding  ROI.  2)  Compute  the  value  of 
P(r)  along  that  line  and  detect  the  boundary  point  as  a  point  at 
which  P(r)  is  minimum.  3)  Repeat  step  1)  and  2)  to  obtain  N- 
detected  boundary  point.  4)  Connect  these  ^(usually  64) -points  to 
form  an  object  boundary.  Repeat  the  process  for  each  ROI.  Note  that 
the  data  must  be  interpolated,  especially  for  small  targets,  to 
acquire  a  sufficient  amount  of  data  for  the  test. 

4.4  -  Iteration  Method 

The  outer  window  size  is  initialized  and  then  iteratively 
determined  to  feed  baok  to  the  double  window  process.  In  the 
boundary  detection  method,  the  length  of  a  spoke  line  is 
initialized  from  the  ROI  boundary,  thus  it  can  be  iteratively 
determined  to  feed  baok  to  the  boundary  detection  process.  For 
both  processes,  these  iterations  update  the  target  size  and  the 
surrounding  background  region. 

5.  -  Multisensor  Fusion Algorithm 

The  availability  of  multisensor  data  raises  the  interesting 
problem  of  how  to  fuse  information  obtained  from  different  sensors. 
The  main  difficulty  is  making  a  deoision  when  the  single-sensor 
algorithms  disagree  with  each  other.  Recall  that  the  hypothesis 
modeled  for  background  distribution  in  different  sensors  all  have 
mean  0  and  variance  of  1,  after  performing  a  double  window  filter 
process  on  the  data.  Therefore,  we  can  fuse  data  from  multiple 
sensors  after  that  process,  but  before  the  detection. 

Futhermore,  segmentations  from  multisensor  (or  different 
segmentors)  can  also  be  fused  using  the  log-likelihood  ratio  test. 

For  example,  suppose  that  A1  and  A2  are  segmented  objects  from 
two  sensors  (or  segmentors) 


Let  B  be  the  box  as  shown  in  Figure  5  and  assume  that  there  exists 
an  optimum  segmented  region  that  satisfies: 

V"  (#V  )nog  ))nog  *  )  is  minimum 
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for  all  SiC  B,  where  #X  denotes  number  of  pixels  inside  region  X 
and  O'X  denotes  standard  deviation  of  pixels  inside  X. 

Then  we  can  apply  the  above  formula  to  fuse  the  subsets  of  hi 
union  with  A2  for  obtaining  a  better  segmentation.  This  is  the 
implementation  of  testing  the  log-1 ikelihood  ratio  test  in  two 
dimensions. 

6.  -  aimilfl&Lan.  Rtaulta  and -Discuss  ion 

Figure  6  illustrates  the  unprocessed  original  laser  radar 
images  and  their  corresponding  boundary  detections.  The  connected 
components  are  ROXs  resulting  from  the  detection  and,  as  we  shall 
see,  the  pixel  detection  is  just  about  60  to  90  percent  of  the 
target  area  but  the  boundary  detection  is  as  good  as  a  hand 
segmentation.  This  is  an  advantage  of  boundary  detection  over 
thresholding  for  processing  a  noisy  image  or  large  deviation  data. 
Figure  7  illustrates  a)  the  original  flir  image,  b)  the 
segmentation  of  a) ,  c)  the  zooming  of  data  from  b)  with  zooming 
factor  of  3  (  also  use  for  d)  and  f )  ) ,  d)  the  zooming  of  data  from 
a)  using  bilinear  interpolation,  e)  the  segmentation  of  d) ,  and  f) 
the  down  zooming  segmentation  from  e) .  The  segmentations  seem  to 
work  well  as  judging  it  by  the  human  eye.  Note  that  at  present 
time,  the  measurement  of  the  segmentation  accuracy  is  still  in 
controversy  because  of  the  difficulty  of  accurately  located  target 
pixels  in  real  images.  For  the  zoomed  case,  the  segmentation  shows 
to  have  an  improvement  over  the  unzoomed  case,  and  thus  for  small 
targets,  data  must  be  interpolated  before  testing  the  hypothesis 
to  deteot  the  boundary. 

However,  most  of  the  algorithms  have  a  failing  case  and  this 
algorithm  fails  if  the  hypothesis  is  false  or  if  the  target  shape 
is  not  a  simple  N-side  polygon.  Therefore,  a  fusion  algorithm  for 
segmentors  is  a  must  for  seeking  an  optimal  segmentation. 

7.  -  conclusions 

This  paper  has  presented  i)  an  iterative  technique  that  uses 
a  double  window  filter  for  target  detection;  ii)  the  implementation 
of  a  log-likelihood  ratio  test  to  refine  the  boundary  of  a  detected 
object  without  requiring  a  threshold.  For  this  latter  case,  the 
concept  of  partitioning  an  object  into  several  subregions  (just  as 
we  have  done  with  a  spoke)  demonstrated  that,  an  object  ought  to 
be  thresholded  at  several  levels,  rather  than  by  a  single  value. 
That  is,  the  object  is  first  estimated  by  segmenting  with  a 
threshold  value,  and  thon  partitioned  into  subregions  to  find  a 
threshold  value  for  each  subregion.  This  concept  should  be 
implemented  by  a  technique  that  uses  a  normalized  threshold  value 
for  segmentation  to  improve  the  performance.  Even  though  this  is 
true,  the  optimal  performance  can  not  be  gained  by  a  single  method 
due  to  the  presence  of  clutter  components  near  the  target. 
Therefore,  a  fusion  algorithm  for  segmentation  is  needed  to  reach 
the  goal,  and  the  log-likelihood  ratio  test  appear  to  be  a  good 
candidate  of  hypothesis  test  for  this  fusion  algorithm. 
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APPENDIX  A 


Proof  of  the  log-likelihood  ratio  test  for  an  "ideal-stochastic" 
case 

Problem  :  let  S  be  a  set  of  random  variables  {sl,s2, . ,sn} 


hypothesis  0  =  91  {sl,s2, . ,sk) 

9  =  92  {sk+1, . ,  sn)  ©1  £  92 


prove  that  y(r)  »  rlog  <r*(r)  +  (n-r)  log  <r*(n-r)  is  minimum  at 


r  *  k  in  these  two  cases, 

case  1  :  01  -  E{sl,s2, . ,sk} 

02  ■  E{sk+1, . .  sn> 


var{sl, s2, . . . . , sk)  «  var {sk+1, . . . . , sn}  *  dj1 


case  2  : 

01  -  var (si, o2,  . 

02  »  var {sk+1, . 

,  ,sn) 

E{sl, s2, 

....,sk)  ■  E{sk+1, . . . . , sn) 

-  E{sl, s2, . 

an)  -  0 

where 

o*l( r)  :  sampled  variance 

o'1  (n-r)  :  sampled  variance 

of  the  set 

of  the  set 

{si, s2, . . . . , sr) 

{ sr+1, . ,  sn) 

0<r<n,  0<k<n,n>0. 

Notice  that  the  meaning  of  the  "ideal-stochastic"  in  this  problem 
is, 

we  assume  that  O'  (r)  ■  O'  (k)  for  all  r  less  than  k 

and  &  (n-r)  -  o'  (n-k)  for  all  r  greater  than  k  . 
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casa  li 


ei  -  £{81,82, 

62  ■  E{Sk+l, . 


.  sk} 
,an) 


var(8l,82, . . .  ,8k)  -  var{8k+l, . . .  ,an)  - 


proof  s 

Sine*  var{«l)  -  ©;*  for  V  i«n  and  E{ai)  f  E{8j)  for  i<k<j 
than,  ©'‘(r)  >  &.L  and  ©‘(n-r)  *  orator  all  ren  . 

at  r  -  k  ,  ©'‘(r)  — >  a?  *k(n-r)  — >  ©,fc 

mmm>  y(r)  ■  r*log  ©k(r)  +  (n-r)log  ©'‘‘(n-r)  la  minimum  at  r-  k 

oaaa  2;  61  -  var{al,a2, . ,ak}  - 

62  -  var{ak+l, . ,an)  -  ©Ja  of4*  oi% 

E(al,s2, .... ,ak)  -  E{ak+1, . . an)  -  E{al, a2 an)  -  0 

y (r)  -  r*log  ©'‘(r)  +  (n-r)*log  o'*,(n-r) 
for  thia  oaaa,  to  prova  y(r)  ia  minimum  at  r  ■  k,  wa  hava  to 
prova  that  dy/dr  -  y'(r)  <0  for  r  <  k 

and  ty'c*-)  >  0  for  r  >  k 

proof  : 

<r‘(r)  -  &,x  for  r  $  k 

-  [k*o',*’+  (r-k)*o;‘j/r  -  *(r)  ,  r  >  k 

©l(n-r)  -  [(k-r)*©^  <n-k)  *o;V(n-r)  -  ^(r)  r  <  k 
-  of  r  >  k 

y(r)  ■  r*log  of1  4-  (n-r)  *log(  ft  (r) )  r  <  k 
-  r*log(  p<  (r) )  +  (n-r)*log  ©J*  r  >  k 
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Abstract 

The  analyiis  of  a  univariate  continuous  sample  is 
proposed  as  a  probability  model  identification  process 
consisting  of  four  steps,  each  of  which  yields  a  distribu¬ 
tion  function,  respectively  denoted  F,  F",  F*,  F~“;  each 
represents  the  outcome  of  successive  steps  in  the  anal¬ 
ysis.  Step  0:  F,  the  true  distribution  function;  Step  1: 
F*,  the  fully  non-paramotric  sample  distribution;  Step 
2:  jF*  m  F(.}F),  a  smooth  distribution  function  obtained 
from  a  parametric  model  F(.;  0)  whose  parameter  S  is  ef 
ficiently  estimated  by  0A;  Step  3:  Comparisons  of  obser¬ 
vations  F“  and  expectations  F*  are  provided  by  suitable 
sample  comparison  density  functions  (denoted  <T)  which 
ase  fully  non»parametric  estimators  of  a  comparison  den¬ 
sity  (denoted  d  or  d(u),,  0  <  u  <  1)  which  compares  F 
and  F*;  Step  4:  F~",  a  smooth- parametric  estimator  of 
F,  is  obtained  from  a  smooth  estimator  <T  which  we  rec¬ 
ommend  as  the  optimal  way  to  provide  a  goodness  of 
fit  test  of  r  to  F*.  Examples  demonstrate  the  insights 
obtainable  from  this  approach. 

1.  Introduction 

We  propose  the  concept  of  unification  of  atatiatical 
methods  in  order  to  develop  a  general  philosophy  of  sta¬ 
tistical  data  analysis.  We  propose  that  ways  of  thinking 
about  atatiatical  enda  (goal*)  and  means  (procedures)  are 
needed  that  provide  a  framework  for  implementing  and 
comparing  aeveral  different  approaches  to  a  data  anal¬ 
ysis  problem.  We  believe  that  unification  haa  benefit! 
which  include:  exiating  (often  parametric)  methods  will 
be  better  understood;  many  new  (often  nonparametric) 
methods  will  be  developed.  The  new  methods  are  usu¬ 
ally  computer  intensive;  consequently  unification  of  sta¬ 
tistical  methods  can  be  considered  to  be  closely  related 
to  computational  statistics.  We  define  computational 
statistical  methods  aa  characterised  by  being  graphics 
intensive  and  number  crunching  intensive. 

This  paper  provides  an  introductory  account  of 
our  approach  to  unification  for  the  case  of  observations 
Yi,...,Yn  which  ore  a  random  sample  of  a  continuous 
random  variable  Y  with  true  unknown  dietribution  func¬ 
tion  F,  quantile  function  F'1,  and  probability  density 
function  /.  The  estimation  of  these  functions  is  called 
the  Probability  Model  Identification  Problem. 

Our  approach  implements  our  favorite  definition  of 
statistics:  Step  1:  make  and  summarise  observations; 


Step  2:  form  expectations;  Step  3:  compare  observations 
and  expectations  (goodness  of  fit  of  model  to  data);  Step 
4:  if  model  does  not  fit  observations,  revise  model  to  fit 
(by  estimation  of  the  comparison  density). 

I  believe  that  one  should  try  to  answer  the  philo¬ 
sophical  question:  what  makes  statistical  inference  possi¬ 
ble  (what  makes  it  poasible  for  ua  to  be  able  to  infer  from 
data  probability  models  that  fit  the  data)?  We  propose 
that  the  answer  includes  the  following  fact:  if  O  is  a  dis¬ 
tribution  function  which  one  considers  aa  a  model  for  the 
true  F,  the  transformed  random  variable  W  <=  G[Y)  has 
distribution  function  F(<3-l(lu))  and  quantile  function 
G(F-1(tt)),  both  of  which  equal  the  identity  function  u 
when  0  o  F  and  F  is  continuous. 

Probability  Model  Identification  Fundamental  Ap- 
prooak:  Base  esthnatkm  criteria  on  the  idea  that  for  a 
continuous  random  variable  the  closeness  of  a  model  G 
to  the  true  F  is  judged  by  measures  which  test  the  hy¬ 
pothesis  that  W  m  0(F)  It  Uniform[0,l]. 

To  test  a  hypothesis  Hq  about  a  random  variable  W 
of  which  one  has  observed  Wu . . . ,  Wn  (a  random  sam¬ 
ple)  early  researchers  (going  back  to  the  first  hypothesis 
testers  such  as  Laplace  (1754)),  proposed  (1)  comput¬ 
ing  a  suitable  test  statistic  T,  (2)  determining  exactly  or 
asymptotically  the  distribution  under  Hq  of  the  statistic 
T\  (3)  using  this  distribution  to  determine  a  rejection 
critical  region  R  of  values  of  the  statistic  T,  assuming 
a  specified  probability  of  rejection  a;  (4)  reporting  re¬ 
jection  of  the  null  hypothesis  Ho  If  the  observed  value 
of  T  belongs  to  R,  or  at  least  reporting  a  p-level  of  the 
observed  value  of  T  (defined  to  be  the  largest  value  of  a, 
probability  of  rejection,  whose  rejection  critical  region 
contain!  the  observed  value  of  the  test  statistic  T).  This 
paper  proposes  to  form  critical  regions  that  represent 
differences  from  1  of  the  probability  density  or  quantile 
deniity  of  tranifomed  random  variables  W  on  the  unit 
interval. 

2.  Parametric  Probability  Model  Identification 

Step  1  of  data  analysis  is  to  form  fully  non¬ 
parametric  estimators:  the  sample  distribution  func¬ 
tion  F“(x)  and  the  sample  quantile  function  Q*(u)  = 
F*-l(u),  where 

F*(x)  =  fractior  of  sample  <  x,  -oo  <  x  <  oo 

The  sample  probability  density  f  does  not  exist 
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a*  an  ordinary  function  but  the  notation  is  used  as  a 
symbolic  function  which  has  meaning  as  an  integrand. 
The  sample  expectation  ET[g{Y)\  is  defined  by 

£T|g(y)l-(l/n)^p(^) 

y«i 

which  can  iilso  be  expressed  as  an  expectation  with  re¬ 
spect  to  F* : 

P\g(Y))  -  [*  g(y)dr(y) 

Step  2  forms  fully  parametric  estimators  F*(*)  and 
Q"( u)  of  the  form 

F*(*) -  F(*|0'), 

QM") 

where  $*  it  an  optimal  estimator  of  the  (vector)  param¬ 
eter  8  of  a  parametric  model  which  we  express  as  a  hy¬ 
pothesis  Ho  given  by 

Ho  :  F(x)  -  F(*jf) 

for  some  value  of  the  vector  parameter  8. 

The  maximum  likelihood  principle  of  estimation 
forms  8"  as  the  value  of  8  maximising  Lm(8),  the  sam¬ 
ple  (average  log)  likelihood,  defined  by 

IV)  «  (2/«)log/(n . Yn;8) 

-(*/»)£;  l«g /(fy#) 

y-i 

Expressions  for  L“(tf)  that  we  find  useful  for  interpreta¬ 
tion  are 

£'(«)-  2£r[log(Kitf)J 

-2  r  log/(y;*)dr(y) 

w  — 00 

"2  j  log /(<?'(«*);  *)<*» 

Sample  likelihood  is  an  estimator  given  the  obser¬ 
vations  of  the  population  likelihood 

L{8)"if  log  /(y;  tfjcf^y) 

*  — 00 

-2  j  log/(Q(u);tf)du 
«-ff(F;F(.i0)), 


defining  cross  entropy  of  two  continuous  distributions  F 
and  G 

H(F-,G)  m  (-2)  (log  p(y))/(y)dy. 

J -OO 

We  define  entropy  of  F  by 

H{F)  -  (  -2)  r  (log/(y))/(y)dy. 

w  —  00 

We  define  Kullback  information  divergence  by 

/(FiO)  -  (-2)  J  ^(log(g(y)//(y))/(y)dy 
-  H(F',G)>-  H(F). 

Important  inequalities  are  l(F;G)  2;  0,H(F;G)  > 
H(F). 

The  procedure  of  maximum  likelihood  estimation 
can  be  expressed  in  terms  of  information  theory  con¬ 
cepts.  The  population  likelihood  is  negative  cross  en¬ 
tropy.  If  the  true  /(y)  equals  /(y;  80)  then  ~L(8)  has 
its  minimum  value  at  8  ■»  0o»  and  the  minimum  value 
is  the  entropy  tf  (F(.;0o))>  Similarly  the  sample  likeli¬ 
hood  L“(6)  has  its  maximum  value  at  8  ■  S',  and  the 
maximum  value  satisfies: 

Lm[8')  m  -H(r,Fr)  S  -ff(F"). 

When  the  parametric  family  /(.;tf)  is  an  exponential 
model, 

L*(8*)  -  me xV(6)  -  -ff(F(.;0*)) 

the  nag-entropy  of  the  fitted  distribution. 

8.  Comparison  density  functions 

For  F  fixed,  minimimislng  population  cross-entropy 
H(F\  F(.;0))  is  equivalent  to  minimising  population  in¬ 
formation  divergence  J(F;F(.;0)).  One  can  regard  this 
as  a  mathematical  measure  of  how  closely  one  can  ap¬ 
proximate  F  by  a  member  of  the  parametric  family 
F(.;  8).  But  we  prefer  a  more  statistical  interpretation 
in  terms  of  the  comparison  density  function  of  two  con¬ 
tinuous  distributions  F  and  G,  defined  as  follows; 

d(u;F,<7  )mgF-i{u)/fF-l(u) 

One  can  show  that 

d(u;F,G)-.D'(uiF,G), 

defining 

D(u:F,G)*>G(F~l(u)). 
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One  can  show  that 

/( F\  G)  =  (-2)  -  log  d{ u  :  F,  G)du. 

Jo 

We  interpret  I(F,  G)  a*  a  meaeure  of  the  cloaeneie  to  1 
of  the  density  d(u  :  F,  G). 

The  information  and  comparbon  deneity  interpre¬ 
tation  of  maximum  likelihood  estimation  that  we  are 
proposing  can  be  expressed  as  follows:  population  likeli¬ 
hood  is  related  to  I(F\  F(.;  0))  which  measures  the  close¬ 
ness  of  F(.;0)  to  F  by  measuring  the  closeness  to  1 
of  d(u\F\F (.',$)),  the  derivative  of  D(u;F,  #,(.;0))  m 
F(F-1(u);0)  which  b  the  quantile  function  of  the  ran¬ 
dom  variable  Wg  «=  F(y;0);  maximum  likelihood  esti¬ 
mation  of  6  b  equivalent  to  finding  the  value  of  6  of  the 
transformation  from  Y  to  Wg  which  b  closest  to  a  Uni- 
form[0,l]  dbtribution  as  measured  by  the  dbtanee  of  the 
sample  quantile  function  from  D(u)  =*  u. 

The  estimation  process  in  Step  2  uses  quantile  func¬ 
tions.  Step  3  uses  distribution  functions  for  goodness 
of  fit  tests;  one  measures  how  close  the  fitted  model 
F(.;  0')  ®  F*(.)  is  to  F~  by  how  close  to  uniform  b  the 
sample  dbtribution  function  D“(u)  ™  F"(F*“  *(«)),  also 
denoted  D"  -  r(QA) 

To  understand  why  the  sample  distribution  function 
b  more  convenient  note  that  it  is  an  estimator  of  D( u)  » 
F(F"-‘(u)  with  derivative 

d(u)«/(r->))/r(r-l(«)). 

An  estimator  cf(u)  leads  to  a  revised  estimator  fAA(y)  by 

r(y)«<f(r(y))r(y). 


their  (asymptotic)  dbtributions  when  parameters  are  es¬ 
timated,  and  they  do  not  satisfy  the  vital  criteria  of  (1) 
looking  at  the  data  and  (2)  providing  insight  into  how 
to  revise  the  model  when  it  does  not  fit. 

The  “components*  approach  chooses  score  functions 
Ji(u) . Jm(u)  satbfylng  for  j  -  1 . m 

/  J,  (u)du  as  0,  f  Jf(u)du  =  1, 

Jo  Jo 

I  Jy(u)  Jj,(u)du  as  0  fOT  j  ^  k. 

Jo 

Components  are  statistics  T  (“linear  detectors*) 
which  are  linear  functionab  in  ef(u)  of  the  form 

r(Jj)  -  ( JjtdT)  -  C  J/(u)cT(u)du  «  fl  Jj(u)dD“(u) 
Jo  Jo 

-U/njEWJ-mW] 

Components  often  can  be  shown  to  be  asymptot¬ 
ically  normal  (under  the  null  hypothesis)  independent 
AT(0,  l/n)  random  variables.  Component  tests  judge  sig¬ 
nificance  of  n'6T"(Jj)  and  nSj,irn,  defining  chi-squared 
test  statbtics 

a-Eim)|8' 

t'mk 

The  Cramer-von  Mbes  goodness  of  fit  test  b  a 
“quadratic  detector*  in  the  sense  that  it  can  be  expressed 
as  a  weighted  sum  of  squares  of  components: 


Step  3  of  our  approach  to  statistical  data  analysis  studies 
various  diagnostics  of  ZT(u)  which  measure  the  signifi¬ 
cance  of  its  difference  from  u,  and  forms  sequences  of 
smooth  approximations  <T(u)  of  the  symbolic  derivative 
cf(u)  Step  4  chooses  an  optimal  <f(u),  which  could  be 
identically  1;  if  this  is  not  the  case,  one  obtains  a  revised 
probability  density  estimator  /**(y), 

4.  Diagnostics  of  comparison  distributions 

The  sample  distribution  function  D“  *  F"(<J*)  of 
W  F"(X)  is  called  a  sample  comparison  distribution, 
estimating  D  **  F[QA)  which  measures  how  well  the  true 
F  is  approximated  by  the  fitted  model  F*. 

Classical  goodness  of  fit  statistics  are  portmanteau 
statbtics,  such  as  the  Cramer-von  Mbes  and  Anderson- 
Darling  tests  in  the  continuour  case  and  the  original  Karl 
Pearson  Chi-Square  test  in  the  discrete  case.  We  do 
not  recommend  them  because  it  is  difficult  to  determine 


r 


(D>)-u)adu-£t^,<r]a 

j-i 


where 

d/(u)  ■*  2  8  cos(;Vu),  w}  *s  1/j'jr. 

The  Anderson-Darling  goodness  of  fit  test  is  a 
quadratic  detector: 

1  (D*(u)  -  u)a/u(l  -  u)du  -  f>3[*,,<r]3 

i'-l 


where 

p,(u)  =  (2;  +  l)'8pj(2u-  1) , uiy  <=  l/j(j  +  1)  B, p,{i) 
are  Legendre  polynomials  on  (-1,1). 
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Hermite  polynomial  goodness  of  fit  test  can  be  de¬ 
fined  by  a  quadratic  detector  with  tuy  =  1/;', 

Mu)  -  (>0“  %(*“>)).  *,•(») 

are  Hermite  polynomiali. 

A  quadratic  detector  can  be  interpreted  a* 

/V(u)  -  u)adu 
Jo 

a  distance  from  1  of  a  emooth  deneity  estimator 
<f  (u)  -1  +  52  <HM«) 

5.  Comparison  density  estimation 

The  novel  eloments  of  this  paper  are  the  role  of  the 
comparison  density  function,  and  especially  the  proposi¬ 
tion  that  estimation  of  the  comparison  density  function 
can  be  used  to  motivate  and  interpret  components.  In¬ 
terpret  sample  components  T"(  Jy)  as  estimators  of 

T{Jj)  -  jf ‘  Jy(u)d(tt)du  -  r  Jy(r(x))dF(*) 

Consider  all  d(u)  obeying  the  constraints  that  for 
j  -  1,.  ..,m 

T(Jy)-r(Jy)j 

one  determines  cfo,m,  defined  as  the  density  obeying  the 
constraints  which  minimises 

IRi(d)  —  log  /  cl^ujiu,  for  A  -  1, 

Jo 

IRo(d)  —2  f  {logd(u)}d(u)du,  fer  A  «■  0, 

Jo 

/JZ-i(d)  —2  /  {-logd(u)}du  for  X  -  -1, 

Jo 

JRx(d)  »{2/A(l  +  A)}  log  J  (d(u)}1+*du  for  A. 

We  call  IRx(d)  Renyi  information  of  index  A. 

An  explicit  formula  for  di,m  is  the  truncated  Fourier 

series: 

iff 

An  exponential  mode!  with  parameters  Si . I’m  i« 

the  form  of  do,m: 

m 

logd0,m(«)  " 


where  ¥  is  the  integrating  factor  that  guarantees  that 
do,m  is  a  density. 

A.  Order  determination 

The  problem  of  goodness  of  fit  can  be  regarded 
as  a  problem  of  determining  the  optimal  order  m*  and 
whether  the  optimal  order  m*  =  0,  As  m  increases, 
do,m(tt)  converges  to  <T(u)  -  d(u;  F*,  F“)  and  IRo(do,m) 
increases:  to 

which  is  the  Moran  goodness  of  fit,  or  non-parametrlc 
entropy,  test. 

The  combination  of  F*(.)  »  F(. ;•*)  and  d0,m-  is 
regarded  as  an  estimator  F~.  A  random  sample  from 
FM  can  be  generated  from  a  random  sample  from  F" 
using  Hotm"  and  the  rejection  method  of  simulation. 

T.  Examples  of  One  Sample  Continuous  Data 
Analysis 

National  Bureau  of  Standards  NB10  Measurements: 
Freedman,  Pisani,  Purves  in  their  textbook  on  Statistics 
(p.94)  report  100  measurements  of  the  10  gram  check- 
weight  NB10  made  at  the  National  Bureau  of  Standards. 
They  report:  “The  normal  curve  does  not  fit  at  all  well. 
The  normal  curve  does  fit  the  data  with  three  outliers  re¬ 
moved.  The  normal  curve  fitted  to  these  measurements 
has  an  average  of  404  micrograms  below  10  grams,  and 
a  standard  deviation  of  about  4  micrograms.  But  in  a 
small  percentage  of  cases,  the  measurements  are  quite 
a  bit  farther  away  from  the  average  than  the  normal 
curve  suggests.  The  overall  standard  deviation  of  0  mi¬ 
crograms  is  a  compromise  between  the  standard  devia¬ 
tion  of  the  main  part  of  the  histogram  (4  micrograms) 
and  the  three  outliers,  representing  deviations  of  18,  - 
30,  and  32  micrograms.  In  careful  measurement  work, 
a  small  percentage  of  outliers  is  expected.  The  only  un¬ 
usual  aspect  of  the  NB10  data  is  that  the  National  Bu¬ 
reau  of  Standards  reported  its  outliers;  many  investiga¬ 
tors  don't.  Realistic  performance  parameters  require  the 
acceptance  of  all  data  that  cannot  be  rejected  for  cause* 

The  NB10  data  illustrates  the  statistical  analysis 
strategy  that  we  propose  be  routinely  applied  to  data. 
Step  1.  Specify  a  parametric  probability  model  for  the 
data  (here  the  model  is  normal).  Step  2.  Estimate  pa¬ 
rameters  of  the  model  (here  mean  and  standard  devia¬ 
tion)  to  be  10  grams-404  mierograms  and  0  micrograms 
respectively.  Step  2*.  Robust  parameter  estimation  by 
Renyi  information  of  index  between  0  and  1  obtains  as 
estimators  of  a  normal  model  (fitted  to  the  part  of  the 
data  that  can  be  well  fitted  by  a  normal  model)  the  same 
mean  and  a  standard  deviation  of  4  micrograms.  Step 


4:  Goodness  of  fit  test  of  normality  by  traditional  tests. 
Step  5:  Maximum  entropy  estimator  of  comparison  den¬ 
sity  d(u;  normal  model,  data)  dearly  indicates  the  na¬ 
ture  of  the  data;  a  poor  fit  of  normal  model  to  data. 
Shape  of  cf(u)  in  interior  of  interval  (0,1)  can  be  inter¬ 
preted  as  expected  curve  if  <f  (u)  estimates 

o'(U;JV(0,(8)a),JV(0,(4)9)) 

*»  xexp  |  -.5  («3  -  l)  (0”1(u))a| ,  k  m  6/4. 

Peaks  of  cf  (u)  it  u  >  0, 1  indicate  longer  tails  than 
normal.  In  general,  one  must  decide  whether  to  con¬ 
sider  these  tails  in  <f(u)  as  outliers  or  as  evidence  that 
a  longer  tailed  distribution  than  the  normal  should  be 
used  to  model  the  data.  In  Figure  1  two  graphs  illus¬ 
trate  the  comparison  density  .estimation  process:  the 
raw  estimator  <T(u)  superimposed  on  a  smooth  esti¬ 
mator  eT(u);  the  exponential  model  smooth  estimator 
do, 4*1  the  orthogonal  polynomial  estimator  di.e*,  and 
a  naive  step  function  estimator  d*  representing  incre¬ 
ments  of  JT(u)  on  8  equal  subintemls.  Diagnostic 
tools  at  step  1  which  help  identify  probability  models 
for  the  data  are  illustrated  by  a  IQQ  plot  of  the  sample 
quantile  function  of  the  data  versus  the  quantile  func¬ 
tion  of  a  normal  with  density  /(«)  —  exp(-»*a),  The 
informative  quantile  function  of  the  sample  is  defined 
on u)  -  (Q»  -  <3%8)}/2{<n.75)  -  Q%M)}. 

Breaking  Strati  of  JBtami  Gheng  and  Stephens 
(1989)  give  a  data  aet  of  breaking  atreas  of  41  beam  spec¬ 
imens  cut  from  a  single  carbon  block  of  graphite  H690, 
and  discuss  goodness  of  fit  tests  of  the  hypothesis  that 
the  data  is  normal.  Let  F(';0*)  denote  the  normal  dis¬ 
tribution  with  maximum  likelihood  estimated  value  of 
0 .  They  show  that  Mornn's  statistic,  which  is  equivalent 
to  /ffo(d(u;ir(*;SA),  JT)  “correctly*  rejects  the  hypoth¬ 
esis  that  the  sample  is  normal,  in  contrast  to  more  tra¬ 
ditional  empirical  distribution  based  statistics  (such  as 
Kolmogorov-Smirnov  and  Oramer-von  Mises)  which  ac¬ 
cept  the  hypothesis  of  normality  for  the  sample  tested. 
The  comparison  density  estimation  approach  indicates 
the  nature  of  the  data;  an  excellent  fit  of  normal  model 
in  interior  of  interval  (0,1)  but  peaks  at  u  »  0, 1  indi¬ 
cate  outliers  or  long  tails  (clearly  evident  in  stem  and 
leaf  table  of  the  data).  One  conjectures  that  a  symmet¬ 
ric  extreme  value  distribution  would  be  a  more  appro¬ 
priate  model.  Figun  2  illustrates  the  comparison  den¬ 
sity  estimation  process  for  a  norma)  model  F(>;0*).  The 
graph  of  D(ti;F('i r),F")  in  graphically  well  fitted  by 
a  uniform  distribution,  and  therefore  passes  traditional 
goodness  of  fit  tests.  The  raw  estimator  d(u;  /’(•;0*),  F') 
is  superimposed  on  a  smooth  estimator.  The  exponen¬ 
tial  model  smooth  estimator  <f(u)  is  superimposed  on 


a  step  function  estimator  computed  from  increments  of 
D( u;  F(>,  9"),  F")  over  8  sub-intervals. 

Cheng  and  Stephens  Break  St  ress  Data 
(Stem  and  Leaf) 
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6.  One  Sample  Discrete  Data  Analysis 

Step  li:  Identify  a  parametric  family  of  probability 
mass  functions  p(x;  0)  to  model1  the  sample  probability 
mass  function  p”(a). 

Step  2:  Parameter  estimation.  Maximum  likelihood 
estimator  0*  can  be  obtained  by  minimising 

/*-i(<*(«i  r, /(■;#))) 

-  (-2)  Mp(*j  * )/!>'(*) }P*(») 

II 

A  parametric  estimator  of  p  is  p"(®)  -  p(*;0‘).  Mini¬ 
mum  chi-square  estimation  uses  the  modified  chi-squared 
distance 

JJMd(u|jr,  *•(.;#)))  -  £<(p(*;0)/p*(*))  -  1)V(») 

• 

Step  3:  Parametric  hypthesis  testing.  To  test  a  hy¬ 
pothesis  Ho  about  the  parameter  0,  let  0#OA  denote  the 
minimum-modified  chi  square  estimator  of  0  under  Ho\ 
equivalent  to  likelihood  ratio  tests  is  the  test  statistic 

/*i(d(u;  r,/(  ;0^#*)))  -  /*,(<*(«;  r./Tj**))) 

Step  4:  Goodness  of  fit  test  of  Ho  :  p  **  p(  ;0*)  or 
equivalently  Ho  :  d(u;/'(';0')|F)  “  0.  Test  the  signifi¬ 
cance  of  the  difference  from  aero  of 

/JMd(u;F(  ;0*),r))  -  /Jl-.s(d(u;r,n;n))- 

Step  5:  Maximum  entropy  goodness  of  fit  tests  and 
estimators  rfo,m*(v)  of  d(u;F(;0*),F)  are  obtained  by 
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minimising  IRo(tT)  among  densities  <f(u)  satisfying,  for 
k  =  1, . . . , m  and  specified  icora  function*  ./*(•:), 

[jht<r\m[jh>dr) 

defining  cT(u)  -  F")  .  For  m  large  enough 

do,m''(u)  equal*  <f  (u)  and  IRo(do,mA)  increase*  to  a  test 
statistic  (alternative  to  that  of  Step  4)  lRo(d(u)  F{-‘,9f), 

ny 

Step  fl:  Rejection  simulation  nonparametric  estima¬ 
tion  of  F.  U»e  an  order  determining  criterion  to  deter¬ 
mine  an  order  m"  with  the  properties;  if  m"  m  o,  ac¬ 
cept  Ho\  ^  one  rejects  Hq  use  doim‘A(u)  as  the  density 
to  be  used  in  the  rejection  method  of  simulating  a  ran¬ 
dom  sample  from  F.  The  combination  of  F('\  $*)  and 
do,m‘A(u)  i»  regarded  as  an  estimator  F~. 
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Raw  cT(u)  to  test  normality,  smooth  by  <T(u) 


Estimators  <T(u;  normal,  data)  Orthogonal  Polynomial; 
Exponential  Model  (graph  closest  to  graph  of  step  function): 


Raw  <T(u)  to  test  normality,  smooth  by  orthogonal  polynomial 


Estimation  ef(u;  normal,  data)  Exponential  Model, 
Step  function 
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ABSTRACT 

The  objective  of  this  program  was  to  determine  the  best  camouflage  net  colors  nnd  texture  for 
Saudi  Arabian  desert  backgrounds.  The  program  began  with  a  fact-finding  visit  to  Saudi  Arabia.  Two 
of  the  findings  of  this  visit  were  that  the  standard  U.S.  Army  desert  net  was  too  dark,  and  that  a 
light-colored  monotone  net  would  best  fit  in  with  the  backgrounds.  Using  this  information,  and 
spectrophotometric  readings  of  soil  samples,  a  series  of  test  nets  were  constructed  Tor  a  field  test  in 
Saudi  Arabia.  Twenty-four  nets  of  different  colors  were  constructed  for  use  in  the  full-scale  test.  The 
standard  U.S.  Army  desert  camouflage  net  was  used  for  comparison  purposes,  Five  test  sites,  repre¬ 
sentative  of  terrain  colors  generally  found  in  the  SANG  operational  areas  around  Riyadh  and  Hofuf, 
were  selected.  The  test  procedure  involved  a  selection  and  ranking  process  as  to  their  ability  to  blend 
with  the  background.  The  nets  were  narrowed  down  to  six  final  candidates,  and  then  these  six  were 
ranked  in  their  order  of  preference.  This  data  was  statistically  analyzed,  and  the  final  color/texture 
recommendations  made.  This  project  joined  the  expertise  of  an  engineer,  statistician,  and  psychologist 
Into  a  working  operational  research  team  to  develop  a  new  camouflage  net  for  SANG. 

1.0  SECTION  I  -  INTRODUCTION 

The  Belvoir  Research,  Development  and  Engineering  Center  (BRDEC)  was  requested  by  the  U.S. 
Army  Project  Manager,  Saudi  Arabian  National  Guard  (PM,  SANG)  to  provide  assistance  In  developing 
a  camouflage  program.  A  fact-finding  team  was  dispatched  to  Saudi  Arabia  to  determine  the  specific 
requirements  for  the  program.  The  primary  areas  of  interest  were  the  vicinities  around  the  capital, 
Riyadh,  and  Daman-Hofuf,  the  center  of  the  oil-producing  area  along  the  Persian  Gulf.  It  was  found 
during  the  visit  that  the  U.S.  standard  desert  camouflage  net  was  too  dark  for  the  light  desert  backgrounds 
of  these  areas,  As  a  result,  a  program  to  develop  a  new  camouflage  net  for  SANG  was  begun. 

Spectrophotometric  readings  of  soil  samples  taken  of  the  areas  of  interest  during  the  fact-finding 
visit,  along  with  color  slides  of  the  areas,  were  analyzed  and  studied  to  determine  a  spectrum  of  desert 
colors  for  use  in  the  construction  of  a  series  of  camouflage  nets  for  a  field  evaluation  in  Saudi  Arabia. 
The  field  evaluation  narrowed  the  spectrum  of  colors.  The  remaining  colors  were  further  refined,  and 
other  colors  were  added  for  a  second  field  test.  Eleven  desert  tones  were  developed  for  the  second 
field  test.  All  the  nets  constructed  for  the  second  field  test  were  monotones  since  it  was  determined 
during  the  first  field  test  that  monotones  blended  better  with  the  Saudi  Arabian  light-colored  deserts. 

In  addition  to  color,  the  variable  of  texture  was  also  investigated.  Texture  is  defined  as  the  degree 
of  perceived  roughness  of  the  camouflage  net.  Tsxture  is  caused  through  the  incision  cuts  in  the  material 
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making  up  the  net.  The  shadows  caused  by  the  incisions  gray  out  the  color  of  the  net.  The  larger  the 
incision  cuts,  the  more  the  net  color  is  affected.  The  incision  size  used  on  the  standard  U.S.  Army  net 
was  too  large  and  tended  to  gray  out  the  light  garnish  colors  which  would  blend  well  into  the  desert 
background.  Two  smaller  incisions  were  developed  for  testing  the  eleven  selected  garnish  colors.  These 
incisions  were  identified  in  this  report  as  "small"  and  "smaller".  This  report  describes  the  second  field 
evaluation  of  the  camouflage  nets  that  took  place  in  Saudi  Arabia,  with  the  objective  of  determining 
the  most  effective  camouflage  net  colors  and  texture  (incision  pattern)  for  use  in  the  Riyadh  and  Hofuf 
regions  of  Saudi  Arabia. 

2.0  SECTION  II »  EXPERIMENTAL  DESIGN 

2.1  Teat  Neta 

The  same  color  was  used  on  pairs  of  nets,  i.e.,  one  for  the  small  incision  size  net  and  one  for  the 
still  smaller  size  incision  net.  In  addition  to  these  nets,  a  two-color  net  (i.e.,  one  color  on  each  aide) 
was  developed  for  each  incision  size  for  evaluation  in  both  light  and  darker  desert  arena.  Two  standard 
U.S.  Army  desert  nets  were  also  included  for  control  and  comparison  purposes,  making  a  total  of 
twenty-six  hexagon  nett.  The  small  incision  nets  were  identified  by  Roman  numerals  I  through  XIII. 
The  smaller  incision  nets  were  identified  by  the  lower  case  alphabetical  letters  a  through  m.  Note  that 
test  nets  V  and  e  had  a  different  color  on  each  side,  and  that  test  nets  VIII  and  h  were  the  standard 
U.S.  camouflage  nets  with  the  standard  incision  size, 

2 2  Teat  Sites 

A  total  of  five  sites  were  selected,  four  at  Riyadh  and  one  at  Hofuf.  The  site  locations  and  color 
designations  were  as  follows: 

•  Site  1- Tan;  Riyadh 

•  Site  2  -  Light  Tan;  Riyadh 

•  Site  3  -  Brownish  Tan;  Riyadh 

•  Sit©  4—  Grayish  Brown;  Riyadh 

•  Site  5  -  Reddish  Tan;  Hofuf 

The  exact  site  locations  were  selected  so  that  the  nets  wero  always  observed  against  the  terrain, 
and  not  highlighted  against  the  sky. 

2 .3  Experimental  Design 

The  nets  were  set  up  at  each  site,  separated  by  incision  size  reading  T  through  "XIII"  for  the 
small  Incision  group  and  "a"  through  "m"  for  the  smaller  incision  group.  A  red  flag  was  placed  on  the 
first,  sixth,  and  tenth  nets,  thus  subdividing  each  of  the  two  incision  size  groups  into  three  smaller  groups 
of  five,  four,  and  four. 

The  test  procedure  involved  a  selection  process  during  which  the  thirteen  nets  from  each  Incision 
sizo  grouping  were  narrowed  down  to  three  final  candidates  within  each  incision  size,  and  the  final 
three  from  each  incision  size  were  grouped  together  and  ranked  in  the  order  of  preference.  The  subjects 
first  selected  three  nets  from  each  incision  size  subgroup  (i.e.,  five,  four,  and  four)  which  best  blended 
with  the  background.  The  results  were  tabulated  in  the  field,  and  the  best  three  nets  were  kept,  with 
the  others  being  taken  down,  This  left  three  subgroups  of  three  nets  each.  The  next  iteration  involved 
selecting  the  best  two  nets  from  each  new  subgroup.  Again  the  results  were  tabulated  in  the  field,  and 
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the  worse  net  for  each  subgroup  was  taken  down.  This  left  a  total  of  sin  nets  standing  for  each  incision 
size  group.  These  six  nets  were  then  rated  as  to  color  blend  and  texture,  with  one  being  the  best  rating 
and  six  the  worst.  The  results  were  tabulated  and  the  three  best  nets  from  each  incision  Rize  setup 
were  identified  and  moved  to  make  one  final  group  of  six  nets.  The  ranking  in  order  of  preference  as 
to  color  and  texture  was  performed  as  before,  to  determine  which  color  and  incision  pattern  best  blended 
with  the  background,  Note  that  this  one-through'six  rating  was  the  first  time  the  small  and  smaller 
incision  size  nets  were  evaluated  as  one  group,  For  all  the  ratings,  described  above,  no  ties  were  allowed. 

A  briefing  was  read  to  the  observers,  at  the  start  of  each  test,  describing  the  sequential  setups  of 
the  nets,  and  pertinent  instructions  for  each  particular  phase  of  the  test.  Ail  evaluations  were  performed 
between  the  hours  of  1200  and  1400  for  proper  sun  angle  and  minimum  shadows.  Each  day  was  clear 
and  hot  with  temperature  ranges  between  118*130  degrees  Fahrenheit,  This  procedure  was  repeated 
for  each  of  the  five  test  sites/ 

2,4  Subjects 

A  total  of  sixteen  male  subjects  (seventeen  for  site  1)  were  used  as  observers  for  each  of  the  five 
sites,  The  subjects  were  screened  for  visual  acuity  using  a  reading  card  produced  by  the  American 
Optical  Corporation,  Pseudo-Isochromatic  Plates  were  used  to  screen  each  potential  observer  for  color 
visioni.  in  order  to  participate  in  this  study,  both  screening  tests  had  to  be  passed,  All  subjects  were 
employees  of  the  Vinnell  Corporation,  stationed  in  Riyadh. 

3,0  SECTION  III  ••  RESULTS 

Of  the  twenty*six  color/texture,  combination  desert  camouflage  nets  tested,  thirteen  advanced  to 
the  final  selection  process  in  at  least  one  of  the  five  sites.  Ifcble  1  summarizes  the  placement  of  these 
nets  for  each  site.  Net  "c"  was  added  only  to  aid  in  computer  data  analysis,  by  completing  the  pairing 
off  of  the  color/texture  combinations.  It  was  not  used  in  the  final  grouping  for  any  of  the  five  sites, 

Table  1 

Final  Placement  of  the  Desert  Camouflage  Nets  for  Each  of  the  Five  Test  Sites 


Nit _ aitll _ Iolil 


1 

2 

3 

4 

5 

1 

No 

No 

No 

Yes 

No 

1 

III 

No 

Yea 

No 

No 

Yes 

2 

V 

Yes 

No 

Yea 

Yes 

No 

3 

IX 

Ye* 

Yea 

Yea 

No 

Yes 

4 

XI 

No 

No 

No 

Yes 

No 

1 

XII 

Yos 

No 

Yea 

No 

No 

2 

XIII 

No 

Yea 

No 

No 

Yes 

2 

a 

No 

No 

No 

YSS 

Yes 

2 

0 

No 

No 

No 

No 

No 

0 

0 

Yea 

No 

Yee 

Yes 

No 

3 

1 

Yet 

Yea 

Yee 

No 

Yes 

4 

k 

No 

No 

Yes 

Yes 

No 

2 

1 

Yea 

Yea 

No 

No 

No 

2 

m 

No 

Yes 

No 

No 

Yes 

2 
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3.1  Net  Color 

The  tame  colors,  with  the  exception  of  nets  V  and  e,  appear  for  each  of  the  two  textures.  Nets 
V  and  e  were  reversed  to  show  their  darker  side  when  used  on  sites  3  and  4  in  order  to  evaluated 
the  side  which  matched  the  background  closer.  Normally,  the  color  perceived  of  a  one-color  net  is 
a  result  of  a  visual  integration  of  the  garnish  color  and  the  incision  shadows  producing  a  third  color. 
In  the  case  of  reversible  nets,  with  different  colors  on  each  side,  the  color  perceived  is  actually  a 
fourth  color,  This  is  the  result  of  the  visual  integration  of  some  of  the  color  visible  from  the  reverse 
side  of  the  net.  Therefore  color  IX  on  one  side  of  net  V  or  e  is  perceived  slightly  altered  by  the 
addition  of  color  XI  on  the  reverse  side  caused  by  the  convoluting  garnish.  It  is  this  "fourth"  color 
that  was  evaluated  during  the  net  tests. 

Using  the  numerical  values  of  one  being  the  best,  six  the  least  preferred,  and  seven  as  not 
being  in  the  final  set  of  nets  for  coior/texture  evaluation  at  each  of  the  five  sites,  the  mean  value 
of  acceptance  with  the  associated  95%  confidence  interval  was  determined.  This  descriptive  data 
is  shown  in  Table  2  and  pictured  graphically  in  Figure  1.  The  higher  the  value  of  preference,  the 
lees  preferred  is  the  net  coior/texture. 

Table  2 


Mean  Preference  and  95-Percent  Confidence  Intervals  for  the 
Final  Desert  Net  Colors,  Small  and  Smaller  Incisions,  Averaged  Across  All  Sites 


Nat 

8ampla 

Standard 

Standard 

06%  Confldtnoa  Intarval 

Color 

Six* 

Maan 

Deviation 

Error 

Lowar  Limit 

llppar  Limit 

I/a 

102 

6.43 

1.2366 

.0970 

6.24 

to 

6.62 

lll/o 

162 

6.38 

1.3606 

.1060 

6.17 

to 

6.50 

V/t 

102 

4.02 

2.0851 

.1638 

4.60 

to 

5.24 

IX/I 

162 

3.60 

2.2881 

.1707 

3.24 

to 

3.06 

Xl/k 

162 

5.01 

1.8776 

.1478 

5.62 

to 

6.20 

XI  I/I 

162 

5.62 

2.0828 

.1636 

5.20 

to 

5.84 

Xlll/m 

162 

6.70 

1.8656 

.1458 

5.41 

to 

5.99 

Colors  IX/i  and  V/e  are  the  most  preferred  colors  with  mean  preferences  of  3.60  and  4.92 
respectively.  The  associated  confidence  interval  states  that  there  is  95%  confidence  that  the  true 
mean  preferences  rest  between  the  upper  and  lower  limits  shown  in  the  table.  An  analysis  of  variance^ 
waa  performed  upon  the  data  in  Thbie  2  to  determine  if  there  were  significant  differences  between 
colors  when  compared  with  sites  (Table  3). 
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Figure  1.  Graphic  Display  of  Table  2 


Table  3 

Analyili  of  Variance  for  Net  Colore  at  to  Blend  With  Sites 


Degrees  of 

8umof 

Mean 

8lgnlf!eanee 

8ouroe 

Freedom 

8quarei 

8quaree 

F-Ratlo 

Level 

Colore 

8 

038,9630 

156.4038 

48.4800 

0.000* 

Error 

1127 

3880.4038 

3.4432 

Total 

1133 

4810.4868 

Bartlett’i  Teit  for  Homogeneoua  Variance* 
Number  Degrees  of  Freedom  -  8 
F  -  15.428  Significance  Level  a  -  0.000* 

♦Significant  at  a  leee  than  0.001  level 
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Table  3  indicates  that  there  were  significant  differences  in  the  ability  of  the  desert  net  colors 
to  blend  with  the  background,  The  Bartlett’s  Test  indicated  that  the  variances  of  each  color  were 
not  homogeneous,  i.e.,  significantly  different,  so  they  are  not  necessarily  from  the  same  population. 

The  Duncan’s  Multiple-Range  Test  (Table  4)  was  used  to  determine  where  these  significant 
differences  in  colors  occurred.  This  test  separates  a  set  of  significantly  different  means  into  subsets 
of  homogeneous  means. 

Table  4 

Duncan’s  Multiple-Range  Test  -  Desert  Net  Colors 


Bait 

Subsat  1 

Subsat  2 

Subsat  3 

Waul 
Subsat  4 

Color  Moan 

Color  Msan 

Color 

Msan 

Color 

Maan 

IX/I  3.60 

V/s  4.92 

Xll/I 

5.52 

lll/c 

6.38 

Xlll/m 

xi/k 

5.70 

5.91 

I/a 

8.43 

Tablo  4  showed  that  the  beat  color  waa  lX/i  for  blending  with  the  desert  backgrounds. 

Table  5  contains  the  nonparametric2''  results  of  the  Kruskal-Wallis  One-Way  ANOVA  as  a  double 
check  upon  the  Duncan’s  Test.  The  Bartlett’s  Teat  indicated  that  the  variances  for  each  level  of 
color  were  not  homogeneous.  Parametric  tests,  such  as  Duncan’s,  assume  homogeneity  of  variance. 


Tables 


Kruskal-Wallls  One-Way  ANOVA  -  Desert  Net  Colors 


Color 

Sampla  8lza 

Moan  Rank 

I/a 

162 

708.72 

lll/c 

162 

704.22 

V/e 

162 

471.28 

IX/I 

162 

308.15 

Xl/k 

162 

638.24 

Xll/I 

162 

575.41 

Xlll/m 

162 

566.49 

Total  1134 

Sampla  8izt 

Chl-Squarad 

8ignlflcancs  Laval 

1134 

161.5826 

0.000* 

Corrsctsd  for  Tits 

8ampla  Slza 

Chi-8quarad 

Significance  Laval 

1134 

223.8419 

0.000* 

'’Significant  at  a  less  than  0.001 

These  results  were  in  agreement  with  Table  3  above. 
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3J  Incision  Site 

Table  6  showed  the  descriptive  data  for  the  small  and  smaller  incisions,  which  is  the  second 
variable  of  interest. 

Table  6 

Mean  Preference  and  95>Percent  Confidence  Intervals 
for  the  Small  and  Smaller  Net  Incisions,  Averaged  Across  All  Sites 


Inolalon  Sample  Standard  Standard  98%  Confidence  Interval 


Size 

Size 

Mean 

Deviation 

Error 

Lower  Limit 

Upper  Limit 

Small 

567 

5.16 

2.30 

.0963 

4.97 

to 

5.35 

Smaller 

567 

5.83 

1.74 

.0731 

5.69 

to 

6.97 

SMALL  INCISION  SMALLER  INCISION 


INCISION  SIZE 


Figure  2.  Graphic  Display  of  Table  6 


Table  6  and  Figure  2  showed  that  the  small  incision  was  preferred  over  the  smaller  incision. 
Table  7  contains  the  analysis  of  variance  performed  upon  the  data  of  Table  6  to  determine  if  these 
differences  are  significant. 
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Table? 


Comparison  of  the  Mean  Preference  of  the  Small  Incision  vs  the  Smaller  Incision 


Dagraae  of 

Sum  of 

Mean 

Significance 

Source 

Fraadom 

Squaraa 

Squares 

F-Ratlo 

Level 

Incision 

1 

128.6808 

128.6808 

01.0538 

0.000* 

Error 

1132 

4690.7760 

4.1438 

Total 

1133 

4819.4668 

Bartlett's  Test  for  Homogeneous  Variances 
Number  Degrees  of  Freedom  ■  1 
F  *  42.300  Significance  Level  a  *  0.000* 

‘Significant  at  a  less  than  0.001  level 


Table  7  indicated  that  the  two  textures,  small  and  smaller  incisions,  differed  significantly 
(a  <0.001)  from  each  other.  The  small  incision  blended  better  with  the  desert  background  than  the 
smaller  incision.  The  Bartlett’s  Tbst  indicated  that  the  variances  of  the  two  textures  were  not 
homogeneous,  i.e.,  significantly  different,  so  they  are  not  necessarily  from  the  same  population. 


Table  8  contains  the  nonparametric  results  of  the  Kruskal-Wallis  One-Way  ANOVA  Test  as  a 
check  upon  the  Duncan's  data.  The  Bartlett's  Tbst  indicated  that  the  variances  for  the  two  levels 
of  texture  were  not  homogeneous.  Parametric  tests,  such  as  Duncan’s,  assume  homogeneity  of 
variance. 


Table  8 

Kruskal-Wallis  One-Way  ANOVA  -  Textures  Small  and  Smaller 


Incision 

Small 

Smaller 


Sample  Size 
687 
887 

Total  1134 


Mean  Rank 

632.04 

602.96 


Sample  Size  Chl-8quered 

1134  13.2978 


Significance  Level 
0.0003 


Sample  8lze 
1134 


Corrected  for  Ties 
Chi-Squared 
16.3926 


Significance  Level 
0.0001 


These  results  are  in  agreement  with  those  of  Thble  7. 

33  Color  and  Iadslons 

It  is  important  to  determine  if  the  variables  color  and  texture  were  independent  of  each  other. 
To  determine  this,  a  two-way  analysis  of  variance  was  performed,  and  the  results  are  seen  in  Table  9. 
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Table  9 


TVo-Way  Analysis  of  Variance  for  Color  and  Tsxture 


Degrees  of 

Sum  of 

Moan 

Significance 

Sourco 

Fraadom 

8quaraa 

Squaras 

F-RstiO 

Laval 

Colors 

6 

038.963 

156.404 

48.666 

0.000* 

Incisions 

1 

128.681 

128681 

40.016 

0.000* 

Interaction 

6 

150.233 

25.039 

7.780 

0.000* 

Error 

1120 

3801 .680 

3.216 

Total 

1133 

4810.457 

♦Significant  at  a  less  than  0.001 

This  table  verifies  the  results  of  Tables  3  and  6  with  significant  P  values  for  colors  and  incisions. 
It  also  shows  a  significant  interaction  between  colors  and  textures.  This  means  that  for  some  nets, 
the  effect  of  either  color  or  incision  was  more  important  than  for  other  nets. 

3.4  Nets 

The  following  analysis  was  done  to  determine  the  beet  net(s)  in  their  ability  to  blend  with  the 
desert  background.  Table  10  contains  the  descriptive  data. 


Table  10 

Mean  Preference  and  95«Percent  Confidence  Interval 
for  Final  Camouflage  Nets  on  Ability  To  Blend  With  Desert  Backgrounds 


Sample 

Standard 

Standard 

95%  Confldsnoa  Intarval 

Nata 

Size 

Mtan 

Daviatlon 

Error 

Lower  Limit 

Upper  Limit 

Small  Incision 

1 

81 

6.42 

1.2027 

.1336 

6.15 

to 

6.69 

III 

81 

5.76 

1.7141 

.1905 

5.37 

to 

8.13 

V 

81 

4.32 

2.4332 

.2704 

3.78 

to 

4.86 

IX 

81 

2.81 

2.3298 

.2589 

2.30 

to 

3.33 

XI 

81 

6.23 

1.5594 

.1733 

5.90 

to 

6.68 

XII 

81 

5.31 

2.2397 

.2489 

4.81 

to 

5.80 

XIII 

81 

C.25 

2.1362 

.2374 

4.77 

to 

5.72 

Smaller  Incision 

a 

81 

0.44 

1.2748 

.1416 

6.16 

to 

6.73 

c 

81 

7.00 

0.0000 

.0000 

7.00 

to 

7.00 

e 

81 

5.52 

1.4501 

.1811 

5.20 

to 

5.84 

1 

81 

4.38 

1.9558 

.2184 

3.95 

to 

4.82 

k 

81 

5.58 

2.1087 

.2343 

5.11 

to 

6.05 

1 

81 

5.73 

1  9040 

.2116 

6.31 

to 

6.15 

m 

81 

8.18 

1.2694 

.1410 

5.88 

to 

6.44 
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SMALL  INCISION 


SMALLER  INCISION 


CAMOUFLAGE  NETS 


Figure  3.  Graphic  Dlaplay  of  Table  10 

An  analysis  of  variance  was  performed  upon  the  data  in  Thble  10  to  determine  if  there  were 
significant  differences  between  the  camouflage  nets  in  their  ability  to  blend  with  the  desert  back¬ 
grounds. 


Table  li 


Analysis  of  Variance  of  Nets  as  to  Blend  with  Sites 


Degree*  of 

Sum  of 

Mean 

Slgntflcane* 

Source 

Freedom 

Square* 

Square* 

F-Ratlo 

Level 

Nutt 

13 

1217.8766 

93.6828 

29.1330 

0.000* 

Error 

1120 

3601.5803 

3.2167 

Total 

1133 

4819.4668 

Bartlett’s  Test  for  Homogeneous  Variances 
Number  Degrees  of  Freedom  -  13 
F  «  9.120  Significance  Level  a  -  0.000* 

•Significant  at  a  less  than  0.001  level 

Table  11  indicated  that  there  were  significant  differences  in  the  ability  of  the  final  desert  nets 
to  blend  with  the  background.  The  Bartlett's  Test  indicated  that  the  variances  of  each  net  were  not 
homogeneous,  i.e.,  significantly  different,  so  they  were  not  necessarily  from  the  same  population. 

The  Duncan’s  Multipie-Range  Test  (Table  12)  was  used  to  separate  this  set  of  significantly 
different  means  into  subsets  of  homogeneous  means. 

Table  12 

Duncan's  Multiple-Range  Test  -  Final  Camouflage  Nets 
fill!  Worat 


8ube*t  i 

8ube*t  2 

Subeet  3 

Subset  4 

8ubs*t  S 

8ube*t  6 

Subset  7 

Net 

Mean 

Net  Mean 

Net  Mean 

Net  Mean 

Net  Mean 

Net  Mean 

Net  Mean 

IX 

2.81 

V  4.32 

XIII  5.25 

k  5.58 

l  5.73 

m 

6.16 

I 

6.42 

1  4.38 

XII  5.31 

1  5.73 

III  5.75 

XI 

6.23 

a 

6.44 

e  5.52 

III  5.75 

m  6.16 

I 

6.42 

c 

7.00 

k  5.58 

m  6.16 

XI  6.23 

a 

6.44 

1  5.73 

III  5.75 

Net  IX  was  the  most  preferred  by  the  ground  observers  as  to  its  ability  to  blend  with  the  desert 
backgrounds.  Nets  V  and  i  were  in  the  second-best  group.  Note  that  net  IX  and  1  were  the  same 
color.  The  only  difference  between  the  two  is  the  size  of  the  incision.  Net  IX  has  small  incisions, 
while  net  1  has  smaller  incisions. 

Table  13  contains  the  nonparametric  results  of  the  Kruskal-Wallis  One-Way  ANOVA  as  a  double¬ 
check  upon  the  Duncan’s  Tbst.  This  was  done,  because  the  Bartlett’s  Test  indicated  non-homogeneity 
of  variance  and  parametric  tests,  such  as  Duncan’s,  assume  homogeneity  of  variance. 
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Table  13 


Kruskal-Wallis  One-Way  ANOVA--Desert  Nets 


Net 

Sample  81m 

Mean  Rank 

1 

81 

707.63 

III 

81 

597.93 

V 

81 

422.30 

IX 

81 

244.13 

XI 

81 

692.50 

XII 

81 

558.17 

XIII 

81 

501.58 

a 

81 

709.80 

0 

81 

810.50 

a 

81 

520.25 

1 

81 

372.17 

k 

81 

583.98 

1 

81 

592.65 

m 

fil 

631.41 

Total  1134 

8ampl»  8lze 

Chl-Squarad 

Significance  Laval 

1134 

210.7103 

0.000* 

Corraetad  for  Tlee 

8ampla  81m 

Chl-Squarad 

8lgnlfloanoe  Laval 

1134 

270.8542 

0.000* 

'Significant  at  a  less  than  0.001 

These  results  are  in  agreement  with  Table  11. 

4.0  SECTION  IV  -  DISCUSSION 

Table  1  identified  the  seven  nets,  their  color,  and  type  of  incision,  that  made  the  final  six 
preferred  nets  for  at  least  one  of  the  five  desert  sites,  on  their  ability  to  blend  with  the  background. 
An  inspection  of  Tsbles  2-5  Indicated  that  color  IX/i  was  the  most  preferred  color  in  the  ability  to 
blend  with  the  desert  backgrounds.  It  was  significantly  (a  <0.001)  better  than  the  second  most 
preferred  color  V/e.  The  data  in  Thbles  6-8  indicated  that  the  small  incision  was  significantly 
(a  <0.001)  preferred  over  the  smaller  incision  in  blending  with  the  desert  background.  Table  9 
shows  a  significant  (a  <0.001)  interaction  between  the  variables  of  color  and  incision.  This  is  in¬ 
terpreted  to  mean  that  for  some  nets,  the  effect  of  color  and/or  incision  size  is  more  important  than 
for  other  nets.  All  colors  are  going  to  appear  even  lighter  when  textured  with  the  smaller  Incision. 
The  smaller  the  incision  size,  the  more  garnish  material  is  visible,  and  the  less  "graying"  out  of  the 
color  will  result,  due  to  the  smaller  shadows.  Conversely,  the  larger  the  incision  cut,  the  less  garnish 
surface  is  visible,  and  the  more  "graying"  out  of  the  visible  color  surface  by  the  effect  of  shadows. 
The  effects  of  the  interaction  of  color  and  texture  are  also  seen  in  Tables  10-12,  which  ideutify  which 
camouflage  nets  were  the  most  preferred  in  blending  with  the  desert.  Net  IX  is  the  same  color  as 
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net  i,  only  it  has  the  small  incision  cut,  while  net  i  has  the  smaller  incision  cut.  Yet  net  IX  is  all 
alone  in  the  group  that  best  blended  with  the  desert  bad  ground.  Net  i  is  in  the  second-best  group, 
along  with  net  V,  which  has  the  small  incisions.  Net  e,  which  placed  in  the  third  group,  had  the 
same  color  as  net  V,  only  it  was  textured  with  the  smaller  incision  cut.  For  darker  colors  such  as 
Xl/k,  the  smaller  incision  cut  created  a  lighter  overall  color  and  enabled  the  net  to  blend  better 
with  the  desert  than  the  small  incision  cut. 

The  parametric  Duncan’s  Multiple-Range  Tb&ts  and  the  nonparametric  Kruskal-Wallis  One-Way 
ANOVA  tests  wore  in  close  agreement  with  each  other.  This  occurred  even  though  the  Duncan’s 
Multiple-Range  Test  yielded  a  significant  F  value,  Previous  work  by  Neubert  et  at3/  had  found 
similar  results.  Thus  it  appears  that  the  assumption  that  the  variances  must  be  homogeneous  for 
parametric  statistics  can  be  overlooked  without  obtaining  invalid  data. 

5.0  SECTION  V  «  SUMMARY  AND  CONCLUSIONS 

Eleven  colors  were  selected  for  field  evaluation  in  Saudi  Arabia.  Two  incision  sizes,  small  and 
smaller,  were  also  investigated.  A  total  of  twenty-four  nett  were  made  for  this  study.  Twelve  of 
the  nets  had  the  small  incision,  and  twelve  nets  had  the  smaller  incision.  The  same  color  was  used 
on  pairs  of  nets,  i.e.,  one  for  the  small  incision  and  one  for  the  still  smaller  incision.  In  addition 
to  these  nets,  a  two-color  net  (i.e.,  one  color  on  each  tide)  was  developed  for  each  incision  size  for 
evaluation  in  both  light  and  darker  desert  areas.  Two  standard  U.S.  Army  desert  nets  were  alto 
included  for  control  and  comparison  purposes,  making  a  total  of  twenty-six  hexagon  nett.  The 
small-incision  nets  were  identified  by  Roman  numerals  I  through  XIII,  The  amaller-incislon  nets 
were  identified  by  the  lower-case  alphabetical  letters  a  through  m.  Test  nets  V  and  e  had  a  different 
color  on  each  side,  and  nets  VIII  and  h  were  the  standard  U.S.  camouflage  nets.  A  total  of  five 
different  sites  were  selected  in  Saudi  Arabia.  They  were  viewed  by  sixteen  male  subjects  (seventeen 
for  site  1).  The  nets  were  evaluated  on  their  ability  to  blend  with  the  desert  background.  A  statistical 
analysis  of  these  ratings  produced  the  following  conclusions: 

A.  The  most  effective  color  was  IX/i  for  blending  with  the  desert  background. 

D.  The  small  Incision  produced  a  net  that  blended  better  with  the  desert  background.  An 
exception  to  this  finding  can  occur  when  the  net  color  is  dark,  such  as  color  Xl/k  (Tables  10  and 
12). 

C.  Net  IX  was  the  most  effective  in  blending  with  the  desert  background. 

6.0  SECTION  VI  --  EPILOGUE 

As  stated  in  Section  I,  this  study  was  done  for  SANG.  The  standard  U.S.  Army  desert  net, 
identified  in  the  test  as  nets  VIII  and  h,  remained  in  use  with  U.S.  forces  with  minor  changes. 
However,  with  the  advent  of  Desert  Shield,  the  U.S.,  based  on  the  results  of  the  SANO  program, 
began  a  special  purchase  of  net  IX  for  shipment  to  Saudi  Arabia. 


123 


REFERENCES 


1.  Natrella,  Mary  G.,  Experimental  Statistics,  National  Bureau  of  Standards  Handbook  91,  U.S.  Depart¬ 
ment  of  Commerce,  Washington,  D.C.,  1966. 

2.  Siegel,  Sidney,  Nonparametric  Statistics  for  the  Behavioral  Sciences ,  McGraw-Hill  Book  Company 
Inc.,  1956. 

3.  Neubert,  Christopher  J.,  Anitole,  George,  and  Johnson,  Ronald  L.,  Comparison  of  Parametric  Versus 
Nonparametric  Evaluation  upon  Non-Homogeneous  Field  Data,  Twenty-eighth  U.S.  Army  Operations 
Research  Symposium,  Fort  Lee,  Va.,  10  October  1989. 


m 


Maintaining  lea  :uental  Optimality  when  Building 
Shorter*.  Fuclldean  Tours 


T.M.  Cronin 

CECOM  Center  for  Signals  Warfare 
WarrentonVA  221 86-5100 


Abstract. 

Reported  upon  are  experimental  runs  of  an  algorithm  designed  to  maintain  incremental  optimality  when 
building  tours  for  the  Euclidean  traveling  salesman  problem.  Unlike  the  Lin-Kemighan  edge  exchange  or  Padberg- 
Rinaldl  branch-and-cut  techniques  which  begin  with  subopdmal  tours  and  proceed  by  iterating  in  an  attempt  to 
converge  upon  or  exceed  the  Held-Karp  lower  bound,  the  new  algorithm  strives  to  maintain  optimality  as  each  city  is 
inserted.  In  previous  Army  research  at  the  CECOM  Center  for  Signals  Warfare,  proofs  were  obtained  to  show  that 
the  underlying  search  space  for  the  Euclidean  traveling  salesman  problem  is  piecemeal  quartic  and  hyperbolic.  To 
exploit  this  new  knowledge,  the  author  has  developed  a  dynamic  programming  algorithm  which  begins  with  a 
baseline  tour  consisting  of  the  outer  (inner)  convex  hull  of  cities,  and  proceeds  by  adding  a  city  at  a  time  to  tho 
interior  (exterior).  How  the  city  is  inserted  into  the  existing  tour  is  dictated  by  a  set  of  quaitic  and  hyperbolic  loci 
which  separate  existing  and  hypothesized  subtours  from  each  other.  The  insertion  may  involve  throe  different  non¬ 
linear  operations:  hyperbolic  extension,  quartic  shunting,  and  quartic  Interchange.  To  test  tho  efficacy  of  these 
operators  with  regard  to  type  1  and  2  statistical  errors,  the  algorithm  as  currently  implemented  is  run  against  a 
benchmark  of  city  databases  for  which  the  optimal  tours  are  known.  For  those  runs  which  result  in  a  suboptimal 
solution,  an  explanation  is  sought  to  facilitate  Axes  to  the  formal  design  specification,  and  the  code  is  subsequently 
changed.  In  this  paper,  the  most  recent  set  of  runs  is  analyzed  and  reported  upon,  and  a  prognosis  for  scaling  up  to 
large  databases  is  forecast.  The  theory  predicts  that  a  run  should  consume  time  as  a  function  of  n3,  where  n  Is  the 
number  of  cities;  this  bound  is  checked  empirically  by  plotting  city  size  vs.  CPU  time  for  several  databases. 


Background. 

The  Euclidean  traveling  salesman  problem  [ETSP]  is  a  long-standing  problem  in  optimization,  having 
roots  and  primary  development  in  the  Held  of  operations  research,  with  ancillary  developments  in  the  fields  of 
computational  geometry  and  graph  theory.  As  is  the  case  with  many  obtuse  problems  in  mathematics,  the  ETSP 
may  be  succinctly  stated.  Given  a  set  of  cities  and  the  distances  between  each  pair,  find  the  shortest  tour  which 
visits  each  city  exactly  once,  except  the  start  city,  which  is  revisited  at  tour's  end.  A  tour  is  simply  a  closed  loop 
connecting  all  the  cities;  the  formal  mathematical  name  for  a  tour  is  a  Hamiltonian  cycle.  One  of  the  interesting 
facts  discovered  early  on  is  that  a  tour  is  not  permitted  to  cross  itself  [FI],  There  are  (n-l)l  /  2  possible  tours 
through  n  cities,  which  is  a  combinatorial ly  prohibitive  number  of  operations  to  perform  by  brute  force,  so  it  is 
therefore  desirable  to  find  an  algorithm  which  arrives  at  a  solution  in  polynomial  time.  The  ETSP  is  a  special  case 
of  the  general  traveling  salesman  problem,  the  foimcr  bearing  the  distinction  that  the  metrics  involved  ore  Euclidean 
distances  rather  than  arbitrary  costs  or  weights. 

To  date,  the  Euclidean  traveling  salesman  problem  remains  unsolved.  By  "unsolved",  it  is  meant  that  no 
one  has  developed  a  formal  proof  of  optimality  for  a  polynomial-time  algorithm  guaranteed  to  produce  the  shortest 
tour.  In  the  mid-seventies,  it  was  proven  that  the  ETSP  is  NP-hard  [01].  This  is  a  somewhat  less  damning 
complexity  result  than  that  obtained  for  the  general  traveling  salesman  problem,  which  belongs  to  the  NP-complete 
class  of  problems  [02].  There  have  been  two  camps  of  researchers  working  on  the  Euclidean  version  of  the  problem, 
with  the  earliest  computational  work  dating  back  to  the  end  of  the  second  world  war  [L2].  The  first  camp  has  striven 
to  produce  an  exact  solution  to  the  problem,  and  In  doing  so  has  pioneered  advances  in  the  field  of  linear 
programming,  including  such  techniques  as  the  simplex  algorithm,  branch-and-bound,  and  branch-and-cut  [PI].  An 
exact  approach  favas  precision  at  the  cost  of  performance.  The  second  camp  of  researchers  has  settled  for  an 
approximate  approach,  by  resorting  to  heuristics  which  produce  high  quality  solutions  per  unit  of  processing  time. 
The  principal  heuristic  techniques  are  k-opt  edge  exchange  (the  most  advanced  of  which  is  the  iterated  Lin- 
Kernighnn),  simulated  annealing,  genetic  algorithms,  elastic  bands,  and  neural  nets  [Jl],  Generally,  the  approximate 
techniques  develop  a  solution  with  more  speed  than  exact  approaches,  at  the  cost  of  precision.  However,  even  this 
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generality  is  moot,  because  some  of  the  heuristic  approaches  render  solutions  orders  of  magnitude  faster  than  others, 
with  only  marginally  inferior  solutions. 


Applications. 

There  are  a  myriad  of  applications  for  the  traveling  salesman  problem.  Among  them  are  job  scheduling, 
resource-constrained  scheduling,  optimal  component  placement,  minimal  hookup  wire,  lowest  transmission  power, 
and  path-constrained  network  flow  [L2,  Jl],  The  structural  similarities  between  the  problems  may  be  somewhat 
subtle.  For  example,  to  map  the  traveling  salesman  problem  onto  job  scheduling,  the  names  of  the  cities  are 
replaced  by  job  names,  and  the  costs  between  cities  are  replaced  by  the  job  setup  times  between  respective  jobs,  The 
job  times  themselves  are  considered  to  be  constants;  the  setup  times  between  jobs  turn  out  to  be  the  crucial  factor. 

Some  of  the  applications  have  been  shown  to  be  NP-complete  problems  (e.g.,  resource-constrained 
scheduling  and  path-constrained  network  flow),  and  therefore  have  failed  to  yield  to  polynomial-time  algorithms 
[02],  Thus,  It  would  seem  preferable  at  this  point  in  time  to  approach  such  problems  with  approximate  techniques. 
However,  in  the  long  term,  if  a  polynomial-time  exact  solution  may  be  obtained  for  the  Euclidean  traveling 
salesman  problem,  then  it  may  be  feasible  to  map  the  resultant  algorithm  onto  one  of  the  harder  problems  in  such  a 
way  that  a  high  quality  (albeit  suboptitnal)  solution  Is  achieved.  This  mapping  is  arguably  most  suitable  for  that 
class  of  problems  for  which  the  triangle  inequality  is  valid. 


Verifying  the  Optimality  of  a  Tour. 

To  test  a  ETSP  algorithm  (whether  it  be  exact  or  approximate)  against  large  databases,  It  Is  necessary  u> 
have  at  hand  some  technique  to  verify  an  optimal  solution  in  polynomial  time.  For  city  databases  of  size  our. 
hundred  or  less,  It  is  possible  to  use  a  variant  of  branch-and-bound  to  check  optimality  in  reasonable  computer  lime 
[Jl].  However,  when  n  becomes  much  larger  than  one  hundred,  certifying  optimality  begins  to  consume 
unreasonable  amounts  of  time.  It  it.  for  this  reason  that  a  technique  based  on  computing  a  lower  bound  on  optimal 
tour  length  has  been  developed  [HI],  This  quantity,  known  as  the  Held-Karp  lower  bound,  Is  computable  in 
polynomial  time,  and  empirical  results  indicate  that  it  is  consistently  within  two  percent  of  optimal  [Jl].  Scientists 
in  the  field  of  operations  research  have  made  good  use  of  the  bound.  Rather  than  strive  for  an  optimal  tour, 
researchers  instead  attempt  to  come  within  a  reasonable  neighborhood  of  the  Held-Karp  bound. 


The  Discovery  of  the  Non-linear  Search  Space  for  the  ETSP. 

Despite  over  forty  yean  of  intense  study  by  computer  scientists  and  operations  research  analysts,  the  search 
space  for  the  Euclidean  traveling  salesman  problem  remained  unspecified  as  of  1990  (i.e.,  it  was  not  known  whether 
the  mathematics  of  tour  construction  was  linear,  non-lincar,  or  transcendental  in  the  number  of  cities).  This  lack  of 
knowledge  prompted  the  author  to  conduct  experiments  during  the  winter  of  1990,  in  an  attempt  to  characterize  the 
space  by  leveraging  the  recently  developed  Held  of  computational  geometry  upon  die  problem.  In  1968,  researchers 
at  the  Johns  Hopkins  University  reported  upon  a  slight  modification  to  a  theorem  due  to  Barachot  to  show  that  an 
optimal  tour  must  preserve  the  order  of  the  convex  hull  of  cities  -  the  shortest  tour  must  contain  these  cities  in  the 
order  in  which  they  appear  about  the  perimeter  [B 1,  B2|.  This  fact  suggested  that  an  experiment  which  Inserts  an 
arbitrary  city  into  a  hull  could  serve  as  a  valuable  testbed  in  which  to  discover  the  geometric  locus  of  equal  hull 
perturbation.  A  perturbation  is  a  subtour  which  leads  Into  the  interior  of  the  hull  through  two  adjacent  hull  vertices, 
to  capture  cities  which  do  not  lie  on  the  hull.  In  conjunction  with  a  perturbation  we  introduce  the  elliptic  distance 
between  a  segment  and  a  point  p,  which  is  defined  to  be  the  sum  of  the  distances  from  the  endpoints  of  the  segment 
to  p,  minus  the  length  of  the  segment. 

When  comparing  a  perturbed  hull  segment  against  another  perturbed  segment,  one  Is  actually  comparing  a 
confocal  system  of  ellipses  against  another,  under  a  continuous  spoctrum  of  elliptic  distances.  The  foci  of  the  two 
families  of  ellipses  are  respectively  the  two  endpoints  of  the  hull  segments  being  perturbed.  In  Army  research  at  the 
CECOM  Center  for  Signals  Warfare  performed  during  the  1990  fiscal  year,  it  was  discovered  that  the  search  space 
Induced  by  the  intersection  of  the  two  confocal  systems  of  ellipses  is  in  general  fourth  order  (quartic),  and  in  special 
cases  hyperbolic  [C2],  The  same  non-llncar  behavior  is  manifested  as  more  cities  are  added  to  the  interior,  which 
means  that  the  general  search  spoce  is  piecemeal  fourth  and  second  order  regardless  of  the  number  of  cities  added  to 
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the  tour  from  within  the  hull.  Dynamic  programming  immediately  suggested  itself  as  an  approach  to  the  problem 
which  might  provide  the  framework  to  keep  track  of  the  quartic  and  hyperbolic  boundaries  of  equal  tour  perturbation 
when  a  new  city  is  added  to  the  existing  space.  Armed  with  the  the  new  information  about  the  non-iincar  search 
space,  the  author  has  proceeded  to  develop  a  dynamic  programming  algorithm  to  maintain  incremental  optimality 
when  building  shortest  Euclidean  tours. 


A  Dynamic  Programming  Algorithm  for  the  ETSP. 

The  algorithm  is  based  on  a  principle  of  incremental  optimality;  the  shortest  tour  containing  k  cities  is  a 
quartic  and  hyperbolic  function  of  the  shortest  tour  containing  k-1  cities.  Beginning  with  a  baseline  tour  consisting 
of  the  convex  hull  of  cities  (the  smallest  bounding  polygon  containing  ail  of  the  cities),  one  city  at  a  time  is  inserted 
into  the  tour  in  an  attempt  to  preserve  the  original  optimality  guaranteed  by  the  hull.  As  currently  specified  by  the 
algorithm,  there  are  throe  types  of  topologies  to  maintain  in  parallel  when  developing  a  tour,  First,  the  tour  may  be 
simply  extended  by  inserting  the  new  city  into  the  space  between  those  two  cities  for  which  the  elliptic  distance  is 
smallest;  this  topology  is  termed  extension  spaa.  A  second  topology  is  one  in  which  the  new  city  causes  a  shunt 
to  be  formed  between  two  existing  perturbations,  to  form  a  new  perturbation  between  the  two  older  ones;  this 
structure  is  called  shunt  spact.  The  final  topology  Is  one  which  deals  with  interchanges  between  perturbations  which 
are  "across  the  hull"  from  each  other;  this  structure  Is  termed  interchange  space.  Extension  space  is  designed  to 
capture  the  hyperbolic  discriminator  inherent  to  extending  an  existing  perturbation,  whereas  shunt  and  interchange 
space  are  models  of  the  quartic  discriminator  instructing  when  to  perform  a  global  merger  of  perturbations. 

A  nested  hull  decomposition  is  computed  during  a  preprocessing  step.  The  decomposition  may  be 
computed  in  0  [n  *  log  n  ]  time,  as  proven  by  Chazolle  [Cl].  Tire  nested  hull  structure,  also  known  as  the  "onion", 
is  devised  to  control  the  order  in  which  the  interior  cities  are  inserted.  To  limit  the  generation  of  greedy 
perturbations,  those  cities  nearest  the  outer  hull  are  installed  first.  The  set  of  hulls  Is  visited  one  at  a  time,  and  each 
hull  Is  traversed  in  a  counterclockwise  fashion,  until  the  set  of  all  interior  cities  is  exhausted,  Therefore  the  order  of 
insertion  is  dictated  by  a  major  key  equal  to  the  ordinal  number  of  the  hull  in  which  a  city  resides,  with  a  minor  key 
equal  to  the  relative  counterclockwise  position  within  the  hull  (N.B.,  there  are  exceptions  based  on  the  angle  which 
the  city  forms  with  the  tour).  An  alternative  strategy  is  to  begin  with  the  innermost  hull  (the  core  of  the  onion)  and 
probe  outwards  one  hull  at  a  time  until  all  exterior  cities  are  processed.  Since  the  quartic  and  hyperbolic  boundaries 
extend  both  inside  and  outside  the  boundary  defined  by  the  current  tour,  the  theory  guarantees  that  it  is  legitimate  to 
process  the  nested  hull  decomposition  in  either  direction,  with  the  same  optimal  solution  produced  regardless  of  the 
processing  order.  An  example  of  bl>diroctional  processing  is  demonstrated  in  the  appendix  for  the  capitals  of  the 
forty-eight  contiguous  states  of  America. 


The  City  Databases. 

Seven  sots  of  data  (Fig.  1)  are  currently  being  used  ai  a  testbed  for  (he  dynamic  programming  algorithm. 
The  first  is  a  ten  city  problem  published  by  Barachet  In  19S7  [Bl],  The  optimal  tour  for  this  small  problem  is 
discussed  and  derived  below.  The  second  set  is  a  sixteen  city  problem  which  appears  in  a  seminal  computational 
geometry  textbook  [P2].  The  third,  fourth,  and  fifth  sets  are  databases  of  twenty,  thirty-seven,  and  forty-one  cities 
which  were  generated  to  exhibit  non-random  behavior;  they  respectively  represent  a  hull  containing  a  single  loop  of 
interior  cities;  a  block  letter  "E";  and  a  block  letter  "S".  For  these  three  databases,  the  shortest  tours  are  not  known 
with  certainty  (insufficient  resources  precluded  certifying  optimality  with  the  branch-and-bound  technique  utilized  by 
the  operations  research  community),  but  it  is  conjectured  that  they  consist  of  the  visually-obvious  structured 
boundaries  of  the  hand-crafted  figures.  The  loop  dataset  is  discussod  below,  and  a  temporal  history  of  the  conjectured 
optimal  tour  Is  contained  in  the  appendix.  The  sixth  dataset  Is  a  forty-eight  city  problem  solved  to  optimality  by 
AT&T  Boll  Laboratories  in  198S  [A  1  ].  The  development  of  its  optimal  solution  Is  also  contained  in  the  appendix; 
both  an  insidc-out  and  outside-in  nested  hull  traversal  are  graphically  portrayed,  with  the  same  optimal  solution 
being  obtained.  The  seventh  and  last  dataset  is  a  one  hundred  twenty-seven  city  problem  formulated  by  the 
University  of  Augsburg  in  1989  [Rl];  this  dataset  has  recently  been  solved  to  optimality  by  the  new  algorithm,  but 
a  detailed  description  of  the  optimal  tour  is  not  included  here,  since  it  will  serve  as  a  primary  example  In  the 
development  of  a  theorem  to  be  published  In  a  forthcoming  paper  [C3J. 

Also  described  below  is  a  set  of  experiments  in  which  eighty  sets  of  cities  are  generated  at  random  to  be 
used  as  databases  to  test  the  analytically-derived  time  complexity  bound  for  the  dynamic  programming  algorithm. 
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For  these  eighty  databases,  the  optimal  tour  lengths  are  unknown  but  are  not  required,  for  the  sole  purpose  in  using 
the  cities  is  to  statistically  test  the  run  time  performance  of  the  current  Implementation  of  the  algorithm,  independent 
of  the  fact  that  the  solutions  developed  may  not  be  admissible. 
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lengths  are  axprtasid  as  a  function  of  pixels  of  the  computer  raster. 


An  Analysis  of  the  Bnrachet  Dataset. 

Figure  2  is  a  visual  graphic  of  a  shortest  tour  evolving  in  time  as  interior  cities  are  incrementally  processed. 
The  data  is  the  Barechet  dataset,  published  in  1937  [Bl],  The  original  constellation  of  cities  is  shown  in  the  upper 
left  comer,  followed  by  a  graphic  of  the  convex  hull,  which  in  this  case  is  simply  a  square.  The  dynamic 
programming  algorithm  then  proceeds  to  add  each  of  the  five  interior  cities  to  the  tour,  The  simple  extensions, 
represented  by  tne  HB  operator,  turn  out  to  be  not  very  interesting.  The  insertions  which  produce  the  most  profound 
changes  an  the  Interchange  operator!,  designated  H,.  As  an  example,  the  last  state  (frame  nine)  ia  produced  by  an 
Interchange.  The  extension  shown  in  the  next-to-lait  frame  causea  the  upper  perturbation  to  yield  two  citiei  to  the 
extended  perturbation  as  at  frame  nine,  wltlle  at  the  same  time  producing  a  new  perturbation  from  the  top,  which  wai 
seen  once  before  at  frame  number  three.  Of  courae,  the  aequenoe  would  look  quite  different  If  the  interior  cities  were 
to  be  Inserted  In  an  aider  other  than  that  dictated  by  the  process  of  nested  hull  traversal,  but  the  final  tour  would  look 
the  tame. 
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An  Analysis  of  ths  Loop  Datassl. 

Figure  3  is  a  tabular  description  of  the  algorithmic  logic  manifested  when  processing  the  twenty  city  loop 
figure  (a  graphic  temporal  history  of  the  logic  is  contained  in  the  appendix).  Although  the  extension  and  shunting 
operations  are  well  represented,  there  are  no  cross-hull  interchanges  which  occur  in  this  database.  The  deferral 
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operations  occur  because  the  city  under  consideration  forms  a  more  acute  angle  with  the  current  tour  than  does  some 
other  sample  in  the  queue  of  interior  cities.  In  such  cases,  the  city  forming  the  more  acute  angle  is  placed  back  in 
the  queue,  and  the  other  city  is  brought  forward  for  processing.  Actually,  the  deferral  and  extension  operations  are 
mundane  when  compared  to  shunts  and  interchanges.  The  two  Interesting  insertions  for  this  dataset  are  the  shunts 
introduced  by  the  addition  of  cities  19  and  112,  both  of  which  radically  alter  the  global  tour  shape.  In  particular,  the 
insertion  of  city  112  causes  the  lower  right  portion  of  the  tour  to  change  from  a  "fishtail"  shape  to  a  concave  loop. 

It  should  be  emphasised  that  the  insertion  of  the  cities  in  some  other  order  might  cause  the  ultimate  loop  behavior  to 
be  displayed  earlier,  but  the  algorithm  is  designed  to  display  the  shortest  tour  for  only  the  cities  which  are  currently 
entered.  A  partial  tour  for  k  cities  may  or  may  not  structurally  resemble  the  shortest  tour  for  all  n  cities. 


Entered  City 


Insertion  Operation 


Relevant  Subtour 


15 

deferral 

• 

119 

extension 

(4,19,20) 

15 

extension 
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m 
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deferral 

■ 
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• 
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deferral 

- 

17 

extension 
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a 

16 
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(4,7,62.19,20) 

112 

deferral 

* 

18 

extension 

(4,8,7,6,5,19,20) 

112 

deferral 

• 

19 

leftsided  shunt 

(3,9,8,7,6,4,5,19,20) 

112 

leftaided  shunt 

(4,5,6,7,8,9,12,13, 

14,13,16,17,18, 

110 

extension 

(4,5,6,7,8,9,10,12,13, 

14,15,16,17,18, 

111 

extension 

(4,5,6,7,8,9,10.11,12,13, 

14,15,16,17,18, 

figure  3.  The  dynamic  programming  result  for  the  twenty  city  loop  figure.  The  Insertions  of 
cities  19  and  113  produce  back-to-back  ahuntlng  operatlous,  tach  of  which  radically  altars  the 
visual  appaarancs  of  tha  optimal  tour.  A  tsmporal  history  of  this  exampls  (ths  loop  datasat)  Is 
contalnad  In  tha  appandlx. 


An  Analysis  of  the  United  States  Capitals  Dataset. 

The  appendix  concludes  with  two  graphics  which  depict  the  development  of  the  shortest  tour  for  the  forty- 
eight  capitals  of  the  contiguous  United  States.  The,  first  graphic  demonstrates  the  same  approach  described  above  for 
the  Barachet  and  loop  datasets:  i.o..  a  baseline  tour  consisting  of  the  outer  hull  is  established,  and  and  the  interior 
cities  are  inserted  incrementally  by  probing  inward  one  hull  at  a  time  until  all  cities  are  exhausted.  The  first 
interesting  behavior  occurs  at  row  three,  column  five,  with  the  introduction  of  Little  Rock:  Oklahoma  City  and 
Jackson  ore  interchanged  into  Little  Rock's  new  perturbation.  Another  interchange  occurs  at  row  four,  column  two, 
when  Frankfort  is  extended  into  Charleston's  perturbation,  which  subsequently  causes  Montgomery  to  be 
Interchanged  from  below.  Yet  another  Interchange  occurs  in  row  four,  column  six,  when  the  introduction  of 
Cheyenne  first  causes  extension  space  to  transpose  Bismarck  with  Pierre,  and  then  forces  the  interchange  of  Salt 
Lake  City.  The  final  Interchange  occurs  in  row  five,  column  three,  when  the  newly  Introduced  city  of  Lansing 
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compels  the  cities  of  Albany  and  Harrisburg  to  be  absorbed  into  Lansing's  perturbation.  By  far,  the  most  dramatic 
behavior  is  encountered  at  row  six,  column  seven,  when  the  introduction  of  SpringficiJ  forces  a  left  shunt. 
Springfield  is  originally  attached  by  extension  space  between  Nashville  and  Frankfort,  but  the  shunt  operator  then 
links  It  to  Jefferson  City  and  synthesizes  a  now  perturbation  issuing  from  the  hull  segment  with  endpoints 
consisting  of  Baton  Rouge  and  Tallahassee.  The  final  tour  shown  in  row  six,  column  2  was  proven  optimal  by 
AT&T  Bell  Laboratories  in  1985  [Al], 

Turning  to  the  second  graphic  in  the  appendix  concerning  the  forty-eight  capitals,  the  alternative  convex 
hull  approach  is  utilized.  This  time,  the  initial  tour  consists  of  the  innermost  hull  in  the  nested  decomposition, 
with  vertices  comprised  of  Des  Moines,  Springfield,  Indianapolis,  and  Columbus.  The  nested  hulls  exterior  to  this 
hull  are  subsequently  processed,  beginning  with  the  one  nearest  to  the  inner  hull.  Because  the  quartic  and  hyperbolic 
loci  remain  valid  regardless  of  the  processing  order,  the  same  optimal  tour  is  ultimately  obtained  at  row  six,  column 
two. 


Some  Remarks  about  the  Augsburg  Dataset. 

The  Augsburg  dataset  consists  of  the  locations  of  one  hundred  twenty-seven  beer  gardens  in  the  city  of 
Augsburg,  Germany.  This  dataset  has  been  solved  to  optimality  by  German  rcsearchors  at  the  University  of 
Augsburg,  using  a  variant  of  branch-and-cut  [Rl].  The  same  optimal  tour  is  obtained  by  the  dynamic  programming 
algorithm  described  in  this  paper.  However,  analysis  of  this  dataset  will  not  be  described  here,  since  it  will  serve  as 
the  primary  example  in  the  development  of  a  theorem  to  be  published  in  a  forthcoming  paper  [C3], 


Scaling  Ups  a  532-City  Dataset. 

A  five  hundred  thirty-two  city  dataset  was  developed  by  Shen  Lin  when  he  was  employed  at  AT&T  Bell 
Laboratories,  and  represents  the  locations  of  AT&T  telephone  offices  in  the  contiguous  United  States.  A  certificate 
of  optimality  has  been  obtained  for  this  data  by  the  originators  of  the  branch-and-cut  algorithm  [PI].  This  database 
is  intriguing  because  it  is  the  largest  database  certified  to  date  for  which  Uie  cities  are  randomly  positioned  in  the 
plane  (a  2392-city  dataset  has  been  solved,  but  the  constellation  of  cities  is  formed  by  repeating  the  same  small 
pattern  of  cities  several  times).  The  dynamic  programming  algorithm  has  not  yet  been  brought  to  bear  upon  tills 
database,  but  it  may  be  feasible  to  describe  the  result  of  its  application  in  the  same  paper  in  which  the  one  hundred 
twenty-seven  city  solution  is  discussed. 


Time  Complexity:  A  Worst-case  Analysis. 

The  dynamic  programming  algorithm  is  continuing  to  evolve  as  a  research  and  development  tool,  and  as 
such  remains  suboptimal.  Nevertheless,  it  is  instructive  to  perform  a  worst-case  analysis  of  the  code  as  currently 
implemented.  A  condensed  algorithmic  flowchart  is  shown  at  Figure  4.  The  label  "In"  is  the  input  loop,  in  which  a 
new  city  is  input  from  the  front  of  the  queue  of  unprocessed  interior  cities.  Upon  entry  from  the  queue,  the  city  is 
processed  by  a  routine  which  checks  for  Intra-perturbation  optimality.  The  new  city  is  first  compared  against  every 
segment  in  the  current  tour  to  discover  die  segment  of  least  elliptic  distance.  This  segment  may  or  may  net  contain 
the  new  city's  nearest  neighbor,  so  a  subroutine  is  called  to  check  the  tour  length  if  an  alternative  hypothesis  allows 
the  connection  to  occur.  To  encourage  the  gradual  Introduction  of  cities  reladvc  to  the  perturbed  hull,  If  some  other 
interior  city  forms  a  larger  angle  with  the  current  tour,  it  Is  brought  forward  for  processing  and  the  candidate  city  is 
put  on  hold.  The  intra-perturbation  routine  concludes  by  reordering  the  city's  perturbation  if  necessary  to  achieve 
optimality. 

Next,  a  global  heuristic  is  applied  to  determine  if  some  'our  segment  forms  a  larger  angle  with  die  newly 
inserted  city  than  the  segment  to  which  it  was  attached  locally  via  the  elliptic  distance  computation.  The  global 
interchange  operator  attaches  the  city  to  such  a  segment  if  it  exists,  and  triggers  a  quadratic  matching  operation  in  an 
attempt  to  absorb  cities  from  other  perturbations.  Next  in  the  processing  sequence  is  the  synthesis  of  the  left  and 
right  shunt  topologies.  The  extended  perturbulion  Is  uttuchcd  to  both  the  nearest  perturbation  on  the  left  and  the 
nearest  perturbation  on  the  right,  and  now  perturbations  are  generated  respectively  to  the  left  and  the  right,  between 
the  perturbations  maintained  by  the  extension  space.  Once  the  shunts  arc  computed,  the  tour  lengths  for  the 
extension  space,  the  loft  shunt  space,  and  the  right  shunt  space  arc  compared,  and  the  minimal  topology  is  preserved. 
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At  this  point,  the  interchange  operator  ia  invoked  once  again  to  absorb  cities  from  other  perturbations,  using  the  left 
and  right  extension  edges  of  the  new  city's  position  as  a  baseline  perturbation.  To  wrap  up  the  processing  of  the 
city,  some  housekeeping  operations  are  performed  to  commit  the  lour  and  its  length  to  computer  memory,  before 
returning  to  die  input  loop  to  process  any  remaining  interior  citiea. 


Figurs  4.  A  high  level  flowchart  of  the  dynamic  programming  algorithm  as  currently 
ImpUmantad.  For  tha  sake  of  brevity,  several  comparison  operations  of  complexity  0[1]  have 
been  omitted.  An  Interchange  operation  (double  box)  la  ulatlvely  expensive)  each  Interchange 
entails  a  quadratic  matching,  four  sorts,  and  four  linear  searches. 


As  the  k*  city  is  processed,  it  is  possible  for  it  to  trigger  two  searches  of  quadratic  complexity,  eight  sorts, 
each  of  complexity  k*log  k,  and  fifteen  searches  of  linear  complexity.  All  of  the  boxes  in  the  flowchart  represent 
processes  of  linear  complexity  or  faster,  except  for  the  two  double  boxes  representing  the  interchange  processes.  The 
interchanges  are  more  expensive,  in  that  they  involve  quadratic  matching  and  sorting  in  a  quest  to  globally  merge 
perturbations  which  may  lie  on  opposite  sides  of  the  hull.  Fim’!'  •  >'  re  is  a  constant  overhead  £0s  associated  with 


the  computer  operating  system  and  hardware  suite.  Therefore,  the  worst-case  processing  time  tw  is  bounded  by  the 
following  cubic  expression: 

tw  ■  I  [2k2  +  8  k  *  log  k  +  IS  k  +  Cos) 

£  2/6  n  *  (n  +  1)  •  (2n  +  1)  ♦  8/2  n  *  (n  +  1)  *  log  n  +  1/2  n  *  (n+1)  +  Cos  n 

«  2  n3  +  4  n2  *  log  n  +  7/2  n2  +  4  n  *  log  n  +  (3/2  +  Cos)  n  . 

Complexity  theorists  refer  to  such  a  bound  as  a  ceiling  function,  because  it  is  derived  empirically  from  an 
algorithm  which  has  not  yet  been  proven  to  be  optimal,  and  in  general  must  be  considered  inferior  to  a  theoretical 
bound  on  performance.  Conversely,  a  floor  function  is  obtained  analytically  from  worst-case  analysis  of  an 
algorithm  known  to  terminate  with  an  optimal  solution  (usually,  once  a  floor  function  is  established  by  theory, 
progress  is  rapid  in  bringing  an  algorithmic  ceiling  function  down  to  converge  upon  the  floor  function),  Since  a 
floor  function  has  to  date  not  been  established  for  the  Euclidean  traveling  salesman  problem,  it  is  necessary  to 
attempt  to  empirically  lower  the  ceiling  function  by  resorting  to  heuristic  techniques,  The  operators  depicted  in  the 
flowchart  are  heuristic  techniques  designed  to  model  the  non-linear  search  space  reported  upon  at  [C2],  The  intrapath 
operators  at  the  left  represent  the  hyperbolic  portion  of  the  locus,  while  those  on  the  right  approximate  the 
discriminator  for  the  fourth-order  components,  Many  of  the  minor  processing  steps  which  are  of  sublinear 
complexity  are  intentionally  omitted,  to  afford  the  reader  as  concise  a  view  as  possible  of  the  global  logic.  The 
author  wishes  to  stress  that  the  implementation  is  at  best  a  stopgap  measure,  which  is  a  useful  research  tool  only 
until  more  geometric  facts  about  the  search  space  become  available.  Indeed,  the  suggested  implementation  is  already 
obsolete,  due  to  a  new  theorem  with  the  potential  to  dispatch  a  significant  portion  of  the  interior  cities  during  a  fast 
preprocessing  step  [C3]. 


An  Experiment  to  Test  the  Validity  of  the  Analytic  Cubic  Bound. 

Tables  1-8  on  the  next  page  are  a  compilation  of  a  set  of  experiments  designed  to  test  the  validity  of  the 
cubic  bound  developed  in  the  preceding  section  of  the  paper.  The  algorithm  as  currently  implemented  was  tasked 
against  sets  of  cities  randomly  distributed  on  a  computer  screen  (the  author  used  the  computer  mouse  to  rapidly  input 
a  set  of  random  points  to  the  screen,  which  were  then  utilized  as  coordinates  for  a  city  database).  The  number  of 
cities  simulated  was  allowed  to  vary  from  ten  to  forty-five,  in  increments  of  five.  For  a  specific  number  of  cities  n, 
ten  sets  of  random  data  of  size  n  were  generated.  Each  set  was  processed  by  the  dynamic  programming  algorithm, 
and  the  following  parameters  were  monitored  by  the  computer  operating  system:  space  (the  number  of  Lisp  cons 
cells,  or  computer  words,  consumed  by  the  run);  lime  (the  number  of  seconds  of  central  processing  unit  time 
consumed  by  the  run);  and  allocation  (the  number  of  seconds  of  CPU  time  devoted  to  dynamic  reclaiming  of 
memory,  using  the  Lisp  garbage  collector). 

Only  the  CPU  time  (the  centra!  column  of  each  dataset)  was  analyzed  statistically.  The  sample  mean, 
variance,  and  standard  deviation  were  computed  for  each  set  of  CPU  time  data.  In  addition  the  best  and  worst  run 
time  outliers  were  selected  for  each  set.  It  was  anticipated  that  the  worst  case  outlier  would  be  a  good  candidate  to 
compare  against  the  cubic  bound  predicted  by  the  analysis. 

Figure  3  is  a  lino  graph  of  the  experimental  results.  For  the  eighty  runs  of  the  algorithm  listed  in  the 
appendix,  the  best-case,  average-case,  and  worst-case  running  times  are  plotted  for  each  of  the  eight  groupings  of  ten 
cities.  Also  included  in  the  same  plot  is  the  cubic  bound;  the  bound  is  computed  for  each  value  of  n,  and  is  scaled 
by  the  constant  .0073  to  render  the  graphic  more  compact  in  the  ordinate  dimension. 

It  is  perhaps  imprudent  to  extrapolate  for  values  of  n  larger  than  those  shown,  but  the  cubic  bound  predicted 
by  the  theory  appears  to  be  a  reasonable  ceiling  function  for  the  worst-case  performance  of  the  algorithm.  Although 
there  is  a  gap  between  the  bound  and  the  worst-case  outlier,  there  are  valid  explanations.  One  explanation  is  that  an 
insufficient  number  of  samples  were  selected  to  see  true  worst-case  behavior.  Another  explanation  is  that  the  author 
was  oveny  conservative  when  conducting  a  worst-case  analysis  of  the  computer  code,  causing  the  cubic  bound  to  be 
somewhat  inflated.  Yet  another  explanation  is  that  a  more  judicious  selection  of  a  scalar  multiplier  of  the  cubic 
expression  could  close  the  gap.  The  important  thing  to  note  is  that  the  bound  is  visually  well-correlated  with  the 
v.  orst-case  plot,  and  that  the  general  behavior  of  the  two  curves  is  markedly  similar. 
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Space  Time  Allocation 


Space  Time  Allocation 


1 

7337 

12.599 

3.380 

2 

5943 

9.739 

3.689 

3 

3271 

5.717 
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4 

2772 

4.879 

5 

2539 

4.342 

1.267 

6 

4726 

8.155 

2.326 

7 

3641 

6.437 

1.579 

8 

3314 

5.714 

9 
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6.021 

1.427 

10 

4192 

7.116 

|  1.399 

n  =  10, 

X  =7.712,  S2 =10.130,  S=3.183 

1 
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16.248 

8.381 

3 

13918 

20.285 
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4 
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18.926 

8.388 

5 
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23.476 

9.775 

6 

15003 

20.873 

9.793 

7 
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13.009 

5.688 

8 
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31.220 

11.403 

9 

9755 

13.709 

6.302 

10 
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19.800 

7.806 

n  =  15.  X  =19.406,  S2=27.906,  S=5.283 

1 

31893 

39.953 

18.919 

2 

39848 

52.327 

23.654 

3 

31961 

41.991 

19.001 

4 

33496 

44.131 

19.732 

5 

28844 

36.980 

18.365 

6 

41600 

50.265 

28.959 

7 

27420 

36.007 

17.767 

8 

27191 

34.494 

17.022 

9 

29502 

37.315 

17.940 

10 

28340 

35.103 

18.456 

n  =  20,  X  =40.857,  S2=39.732,  S=6.303 

1 

63375 

72.603 

38.143 

2 

50002 

47.761 

41.782 

3 

48373 

56.510 

27.231 

4 

52179 

67.577 

30.072 

5 

53730 

65.259 

32.730 

6 

63618 

77.347 

39.047 

7 

67187 

81.024 

40.680 

8 

64343 

78.275 

37.596 

9 

62891 

75.275 

35.368 

10 

49037 

56.832 

28.396 

n  =  25.X 

- 67.846 ,  S2=123.389,  S=1 1.108 

1 

102662 

121.708 

58.073 

2 

101337 

125.221 

61.622 

3 

92058 

98.096 

66.107 

4 

108662 

132.876 

64.330 

5 

104230 

129.015 

62.486 

6 

108849 

134.460 

66.508 

7 

82841 

97.734 

46.839 

8 

103552 

68.177 

119.213 

9 

84362 

133.353 

285.494 

10 

101140 

124.738 

59.313 

n  =  30,  X  = 116538 ,  S2=469.822,  S=21.675 


1 

150861 

185.943 

101.698 

2 

121924 

152.557 

86.844 

3 

161438 

197.643 

114.430 

4 

186448 

246.866 

129.780 

5 

163579 

204.787 

114.469 

6 

161428 

212.644 

112.123 

7 

178192 

227.529 

121.439 

8 

214286 

295.458 

152.270 

9 

180710 

196.557 

105.282 

10 

156222 

167.035 

64.046 

n  =  35,  X  = 208.702 ,  S2 -1671. 621,  S=40.885 


1 

209331 

224.835 

98.215 

2 

241452 

267.132 

99.419 

3 

206614 

212.946 

85.847 

4 

263413 

295.444 

115.531 

5 

295821 

348.702 

136.283 

6 

239564 

257.743 

110.176 

7 

229468 

263.191 

106.129 

8 

223627 

239.093 

97.718 

9 

216796 

249.739 

96.802 

210261 

244.238 

91.558 

n  =  40,  X  =260.306,  S2 =1492.335,  S=38.631 


1 

329234 

389.722 

244.246 

2 

354376 

411.498 

156.177 

3 

382554 

490.126 

179.530 

4 

426650 

593.475 

208.267 

5 

293465 

301.532 

114.221 

6 

404882 

505.157 

173.208 

7 

384530 

530.231 

278.536 

8 

341033 

413.296 

213.390 

9 

335337 

389.147 

196.894 

10 

278591 

344.297 

178.995 

n  =  45,  X  =436.848,  S2  =8147.41 5,  S=90.263 


Tables  1-8.  Space,  Time,  and  Allocation  Complexity  for  Eight  Sets  of  ETSP  Experiments 

tn  Is  the  numher  of  cities  per  experiment;  Space  Is  the  number  of  Lisp  cons  cells  consumed  by  a  run; 
,'imr  Is  the  number  of  seconds  of  (TU  time  consumed  by  a  run;  und  Allocation  Is  the  time  dedicated 
to  the  Lisp  r*ur'  r;e  collector) 
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Best  Case 
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Worst  Case 
Cubic  bound 


Figure  5.  A  line  plot  of  the  time  complexity  of  the  dynamic  programming  algorithm  as  a  function 
of  the  number  of  cities  processed. 


Summary. 

Some  very  preliminary  statistical  experiments  on  the  worst  case  behavior  of  an  algorithm  designed  to 
provide  an  exact  solution  for  the  Euclidean  traveling  salesman  problem  indicate  that  the  run  time  of  the  algorithm  is 
in  fact  bounded  by  an  expression  cubic  in  the  number  of  cities.  The  algorithm  is  based  on  some  recent  theoretical 
results  pertaining  to  the  non-linear  search  space  for  the  Euclidean  traveling  salesman  problem,  and  as  such  the 
computer  code  has  not  yet  reached  an  optimal  level  of  maturity.  Nevertheless,  it  has  proven  useful  to  statistically 
compare  the  performance  of  the  dynamic  programming  algorithm  against  the  cubic  bound  predicted  by  a  cursory 
examination  of  the  currently  implemented  software.  The  worst-case  statistical  outliers  compiled  for  each  set  of 
experiments  are  indeed  bounded  by  the  cubic  expression  developed  analytically  from  the  current  formal  design 
specification  of  the  algorithm.  It  is  apparent  that  the  science  of  statistics  is  invaluable  with  regard  to  gauging  the 
probabilistic  performance  of  an  algorithm  versus  its  analytic  time  complexity  bound. 


Future  Directions  of  the  Research. 

An  entirely  independent  issue  is  whether  or  not  the  algorithm  is  admissible:  i.e.,  whether  it  terminates  with 
an  optimal  solution.  It  is  desirable  to  attempt  to  prove  that  the  dynamic  programming  technique  is  admissible;  a 
proof  by  induction  seems  promising.  Thus  far,  the  implementation  is  proceeding  in  the  spirit  of  the  Hungarian 
mathematician  Lakatos,  who  contended  that  a  theory  is  never  truly  proven  until  sufficient  time  has  passed  such  that 
the  community  at  large  accepts  the  theory,  based  on  the  fact  that  counterexamples  cease  to  be  forthcoming  from 
empirical  testing  [LI  ].  The  implementation  discussed  in  the  paper  is  at  a  stage  where  counterexamples  can  still  be 
found.  However,  the  author  feels  that  the  counterexamples  arc  sufficiently  trivial  to  be  local  rather  than  global, 
which  indicates  that  the  problems  remaining  to  be  ironed  out  arc  details  of  implementation  rather  than  profound 
issues  of  conceptualization.  It  seems  important  to  pursue  the  proof  of  optimality;  otherwise,  the  new  algorithm  will 
be  vulnerable  to  the  same  kinds  of  criticism  which  plague  all  heuristic  approaches  to  problem  solving. 

Ttic  algorithm  will  continue  to  undergo  empirical  testing,  as  the  number  of  cities  is  scaled  up.  A  good 
source  of  benchmarks  is  maintained  at  reference  [RI ].  As  mentioned  above,  a  one  hundred  twenty-seven  city  database 
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has  recently  been  solved  to  optimality  by  the  dynamic  programming  algorithm.  It  it  desirable  in  1991  to  move 
ahead  to  a  five  hundred  thirty-two  city  certified  benchmark  [PI].  Unfortunately,  there  are  only  a  handful  of  large 
databases  for  which  a  certificate  of  optimality  has  been  obtained. 

In  preparation  is  a  paper  which  describes  •  new  geometric  result  pertaining  to  the  aspect  angle  which  an 
interior  city  forms  with  the  convex  hull,  and  the  positive  implication  of  the  result  u  a  preprocessing  step  for  the 
dynamic  programming  algorithm  [C3].  It  is  premature  to  forecast  the  utility  of  the  new  theorem,  but  empirical 
testing  indicates  that  on  the  average  a  surprisingly  large  percentage  of  cities  interior  to  the  hull  may  be  inserted  into 
the  tour  in  a  fast  preprocessing  step. 
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AN  ALGEBRAIC  DERIVATION  OF  VARIANCE 
OF  THE  GEOMETRIC  DISTRIBUTION 

Richard  M.  Brugger 

Product  Assurance  and  Test  Directorate 
U.S.  Army  Armament,  Munitions  and  Chemical  Command 
Rock  Island,  Illinois  61299-6000 


ABSTRACT.  Typically,  the  variance  of  the  geometric  distribution  is 
derivecTusfng  generating  functions  or  moment  generating  functions.  This  paper 
provides  an  alternative  using  algebra  only. 

1.  INTRODUCTION.  Consider  an  unbounded  sequence  of  Bernoulli  trials, 
each  trTaTnhavfrTg  probability  p  of  success  and  probability  q  =  1  -  p  of 
failure.  Let  X  denote  the  trial  number  of  the  first  success.  Then  X  is 
distributed  geometrically. 

The  geometric  distribution  plays  an  important  part  in  many  probabilistic 
considerations.  For  example,  it  is  the  underlying  reliability  distribution 
for  an  item  that  functions  discretely  with  an  assumed  constant  failure  rate, 
such  as  a  gun.  It  also  plays  an  important  part  in  sampling  plans.  Dodge's 
original  work  on  continuous  sampling  plans,  described  below,  incorporated 
geometric  distributions  extensively. 


2.  DERIVATION  OF  DODGE'S  h.  Dodge  (1943)  used  the  following  derivation 
of  a  geometn c^rTstributi on  curtailed  after  i  trials  (where  i  is  fixed,  not  an 
index).  For  the  remainder  of  this  paper  0  <  p  <  1  and  thus  0  <  q  <  1. 


h  =  1  (p+2pq+3pq2+4pq3+. . ,+ipq1  *) 

(la) 

l-q1 

=  p  (l+2q+3q2+4q3+...+iq1_1) 

(lb) 

l-q1 

2  3  i 

=  p  d  ( 1+q+q  +q  +. . .+q  ) 

(lc) 

l-q1  dq 

=  p  d  (l-q1+1)  . 

(Id) 

l-q1  dq  l-q 

Considering  only  the  derivative  portion  yields 

d  (l-qi  +  1)  (l-q)(-(i+l)q1 ) - (l-q 1 +1 ) ( - 1 ) 

(2a) 

2 

dq  l-q  p 

=  p(-i -l)q1+(l-q1+1) 

o 

(2b) 

£ 

P 

=  i-q1 ( i+pi )  • 

(2c) 
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Brugger  (1989)  provided  the  following.  Returning  to  the  first  expression: 
h  ■ _ 1  (p+2pq+3pq2+4pq3+. .  ,+lpq'1"1)  . 

17 

Since  p  -  l-q»  this  becomes 

h  «  1  (l-q+2(l-q)q+3(l-q)qZ+4(l-q)q3+. ,.+i(l-q)q1"1) 

7-7 

■  _ 1  (l-q+2q-2qZ+3qZ-3q3+4q3-4q4+...+1q1"1-1q1 ) 

77 

■  1  (l+q+qZ+q3+q4+. . ,+1q1-1-1q1 ) 

77 


(3a) 

(3b) 


(1) 

(4a) 

(4b) 

(4c) 


The  first  1  terms  within  the  parentheses  follow  a  geometric  progression,  so  the 
above  becomes 


h 


Jbal .  IsL 

p  i-q1 

p  pd-q  ) 


. i  (i-q1 ( i+pi ) ) 

Pd-q1) 


(5a) 

(5b) 

(5c) 


This  Is  the  same  as  Dodge's  result  (see  equation  3b),  but  In  this  latter  case 
only  algebra  was  used  to  obtain  the  result. 


3,  DERIVATION  OF  THE  MEAN  OF  A  GEOMETRIC  DISTRIBUTION.  Using  the  method 
described  "above',  it  Is  straightforward  to  derive  the  mean,  E ( X) ,  of  a  geometric 
distribution: 


£  jpq^1 

(7a) 

j-1 

2  3 

lp+2pq+3pq  +4pq  +. . . 

(7b) 

l-q+2q-2qZ+3qZ-3q3+4q3-4q4+. . . 

(7c) 

2  3  4 

1+q+q  +q  +q  +. . . 

( 7  d ) 
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=  1  a  l 

(7e) 

1-q  P 

4.  DERIVATION  OF  THE  VARIANCE.  By  definition, 
Va r  (X)  -  E(X2)-(E(X))2. 

(8) 

From  equation  7e  above 

(E(X))2  ■  1/p2 

0) 

is  obtained. 

E(X2)  “  2  J2pqJ_1 

J-1 

(10a) 

o  3 

■  p+4pq+9pq  +16pq  +... 

(10b) 

■  l-q+4q-4q2+9q2-9q3+16q3-16q4+. . . 

(10c) 

°  l+3q+5q2+7q3+. . . 

(10d) 

It  is  claimed  that  the  series  in  equation  lOd  will 
equal  to  the  odd  Integers,  This  Is  easily  proved. 
Then 

continue  with  coefficients 

Let  n  be  any  positive  Integer 

n2-(n-l)2  ■  n2-n2+2n-l 

(1U) 

■  2n-l 

(lib) 

where  the  expression  in  lib  is  clearly  an  odd  Integer.  Summing,  we  obtain 

E(X2)  -  2  ( 2 j - 1  )q^ “ 1 

J-1 

(12a) 

«  2  ?  jq^"1  -  Z  q^”1 

J-1  J-1 

(12b) 

-  2  !  Jpqd_1  -  1 

P  J-1  P 

(12c) 

-  2  1  1 
p  •  p  “  p 

(12d) 

-  2  1 

7? "  r 

P  P 

(12e) 

ak£ 

(12f ) 

P 

(12g) 
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Recall  1ns  that 


Var  (X)  -  E(X2)-(E(X))2 

(8) 

it  Is  seen  that 

Var  (X)  »  a+l  _  1 

(j  p^ 

(13a) 

m*x 

(13b) 

5,  EXTENSION  TO  THE  NEGATIVE  BINOMIAL  DISTRIBUTION.  If  one  is  interested 
in  some  positive  integer  n  (n>T)  of  successes  in  a  series  of  Bernoulli  trials 
with  constant  probability  p  of  success,  the  trial  number  of  the  n'th  success  Is 
distributed  according  to  a  negative  binomial  distribution,  which  is  an  extension 
of  the  geometric  distribution. 

Then 

E(X1+X2+...+)Cn)  *  nE(Xt)  (,14a) 

■  n/p  (14b) 

and  Var  (Xj+X2+. . .+Xn)  »  nVar(X1)  ( 15a) 

-  nq/p2  (15b) 
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A  LINEAR  PROGRAMMING  MODEL  FOR 
QUEUEING  IN  OPERATIONAL  AVAILABILITY 
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ABSTRACT .  Operational  Availability  Aq  is  tha  term  used  to 
refer  to  readiness  of  a  military  system  for  battlefield  use.  The 
servicing  of  such  a  system  can  be  divided  into  several  stages,  each 
of  which  may  be  modeled  by  an  Erlang  type  queue  M/Et/B,  s  being  the 
number  of  service  personnel.  Maximum  operational  Availability 
is  equivalent  to  minimization  of  Total  Corrective  Maintenance  Time 
(TCM)  plus  Total  Administrative  and  Logistic  Downtime  (TALDT) . 
Under  appropriate  constraints  the  maximum  Aq  problem  can  thus  be 
formulated  as  a  linear  programming  problem. 

1.  INTRODUCTION.  TRADOC/AMC  Pamphlet  70-11,  RAM  RATIONALE 
REPORT  HANDBOOK,  defines  (p.  E.18)  Operational  Availability  A0  by 
the  following  formula: 

Aq  -  (OT  4  ST) / (OT  4  ST  4  TCM  4  TPM  4  TALDT).  (1) 

Here  OT  *  Operating  Time,  i.e.,  time  during  a  mission  profile  when 
the  system  is  "on"  and  actively  performing  at  least  one 
of  its  missions; 

ST  m  standby  Time,  i.e.,  the  time  during  a  given  period  when 
a  system  is  inoperative  but  is  operable; 

TCM  ■  Total  Corrective  Maintenance  Time,  i.e.,  the  total  time 
spent  on  restoring  a  failed  item  to  oporational 
condition; 

TPM  ■  Total  Preventive  Maintenance  Time  spent  in  activities  to 
maintain  an  item  in  a  specified  condition; 
and  TALDT  ■  Total  Administrative  and  Logistics  Downtime  spent 
waiting  for  parts,  arrival  of  maintenance  personnel, 
or  transportation,  during  a  given  period. 

In  a  tactical  situation  Standby  Tima  ST  and  Preventive  Maintenance 
Time  PCM  can  frequently  be  neglected,  in  which  case  the  above 
formula  reduces  to 

Aq  -  1/[1  4  (TCM/OT)  4  (TALDT/OT)). 

It  is  clear  from  this  expression  that  for  given  OT,  A0  is  maximized 
by  minimizing  TCM  4  TALDT! 

max  A0  -  min  {TCM  4  TALDT). 
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The  right  hand  aid*  of  this  expression  will  provida  the  basis  for 
a  linear  programming  approach  to  optimization  of  Operational 
Availability. 

The  present  approach  was  suggested  by  a  requirement  for 
deciding  among  competing  maintenance  systems  for  a  deployed  IEW 
network  along  the  FLOT  (Forward  Line  of  Own  Troops)  .  The 
maintenance  structure  consisted  of  as  many  as  four  stages:  mobile 
contact  units  (BMCT)  which  provided  minimal  repairs,  usually  in  the 
form  of  simple  replacment  of  defective  line  replaceable  units 
(LRU's),  at  the  FLOT;  shop  repairs  (BS)  at  a  forward  level;  mobile 
contact  units  (LMCT's)  dispatched  from  a  rear  light  electronics 
maintenance  company  to  handle  more  complex  diagnosis  and  repairs; 
and  a  rear  echelon  repair  shop  (LS)  for  major  diagnosis  and 
repair/ replacment.  Associated  with  each  level  is  a  characteristic 
MTTR  (Mean  Time  To  Repair)  and  an  ALDT  (Administrative  and  Logistic 
Downtime)  for  maintenance.  These  parameters  will  play  key  roles  in 
the  queueing  models  for  the  maintenance  process (es).  Both  the  MTTR 
sequence  and  the  ALDT  sequence  may  be  modeled  as  a  four-stage, 
series  Erlang  queue  M/E./s,  where  M  represents  "customers"  (i.a., 
FLOT  system  elements  with  defective  parts  (LRU's))  occurring  at 
random  according  to  a  Poisson  process,  E4  denotes  an  Erlang 
distribution  representing  a  four-stage  FIFO  servioe  protocol,  and 
s  denotes  the  number  of  servers,  i.e.,  maintenance  personnel  at  any 
given  level. 

2. . Eafclmfclan _ at _ Afoilnlitaatlyi _ and _ Locietic _ pawns imw _ far 

Maintenance  fALPTm) .  The  expected  waiting  time  at  a  defective  LRU 
site  for  maintenance  personnel  is  given  by  the  formula  [Qaver  & 
Thompson,  1973,  p.  505] 

E<WalotJ  *  *  (MTTR),  U,/(l  -  U,),  (2) 

where  the  summation  on  i  runs  over  the  4  stages  and  the 
"utilization",  or  "traffic  load  factor",  is  defined  to  be 

U,  -  MTTR,  /  (s,  MTBF)  ,  (3) 

Here  s,  is  the  number  of  "servers"  at  stage  i,  and  MTBF  is  the  Mean 
Time  Between  Failures.  If  u,  turns  out  to  be  greater  than  one,  the 
system  saturates.  Hence  0  <  u ,  <  1  will  be  assumed  in  the  sequel. 
Since  then  u/(i  -  u)  -  u  +  Z  u"  i  u,  Eq.  (2)  may  be  written  in  the 
series  form 

E(WAiDTm>  *  E  (MTTR),  U,  -  (1/MTBF)  E  (MTTR,)2  (1/s,).  (4) 

Apart  from  costs,  the  defining  of  an  organizational  structure 
essentially  resides  in  the  assignment  of  personnel  and  facilities. 
To  utilize  (4)  in  a  linear  programming  formulation  of  optimality 
for  Aq  we  therefore  introduce  as  a  new  variable  x, ,  the  reciprocal 
of  s)r  i.e.,  the  reciprocal  of  the  number  of  servers  at  stage  i. 
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Inequality  (4)  then  becomes 

E<WACDTm>  *  (1/KTBF)  X  (MTTR,)2  X, .  (5) 

The  introduction  of  the  new  variable  x,  makes  possible  formulation 
as  a  linear  programming  problem,  but  it  is  not  an  unmixed  blessing. 
Being  a  reciprocal,  the  variate  x,  will  accentuate  the  tendency  of 
a  linear  programming  problem  to  give  solutions  in  non- integer  form 
and  may  also  lead  to  instabilities  for  small  assignments. 


■i.  oi  Mnini»trativ  ana  loquiip  uownumi  ror  Transport 

ALDTt .  in  the  present  model,  the  Administrative  and  Logistic 
Downtime  awaiting  parts  from  CONUS  is  neglected  since  it  exceeds 
the  hypothesized  time  scale.  it  only  remains  therefore  to 
formulate  ALDTt.  Since  transport  is  involved,  only  the  mobile 
contact  teams,  theBMCT's  and  the  LMCT's  will  contribute.  The  time 
involved  may  be  estimated  by  distributing  tha  total  number  of 
manhours  spent  by  each  in  travel  between  base  and  FLOT  and  equating 
this  to  the  TALDTt  requirement: 

E{TALDTt/F}  -  Probability {service  by  BKCT) • (Mean  roundtrip  time 

from  BN  to  FLOT) X,  4  (6) 

+  Probability (service  by  LKCT}*(Mean  roundtrip  time  from 

LEMCO  to  FLOT) X,* 

These  expected  roundtrip  times  may  be  estimated  either  from  data  or 
by  dividing  the  doubled  distance  by  the  avrrage  speed  of  the  mobile 
contact  unit.  The  probabilities  of  service  by  BMCT's  or  LMCT's  are 
estimated  from  appropriate  decision  trsos. 


4.  Estimation  of  Corrective  Maintenance  Time.  The  average  time 
required  for  corrective  maintenance  is  the  Total  Corrective 
Maintenance  Time,  TCM,  divided  by  the  mean  number  of  failures: 
TCM/F.  In  equilibrium  this  is  the  same  as  the  MTTR  divided  by  the 
number  of  servers  at  any  particular  stage  of  the  4-level 
maintenance  process: 

E{Wcm}  -  X  (MTTR,)  /s,  -  X  (MTTR,)  X,.  (7) 


Function.  Combining  th*  results  of  Eqs.  (4),  (5), 
(6),  and  (7)  leads  to  the  following  objective  function  for  the 
maximization  of  A0: 

min  {TALDT  4  TCM}  «  min  { { (MTTR,)2/MTBF  4  MTTR,  + 

+  distance|l.FWT/kmphMWTl  x,  4 

,  (8) 

+  E,.M[(MTTR,)7MTBF  4  MTTR,  ]X,  4 

+  [(MTTRj)2/MTBF  +  MTTR,  4  distancell.noT/kaphUCT]x,. 


5.  The  Constraints.  It  remains  te  formulate  the  conetrainta.  We 

need  for  thie  purpoee  the  following  result  from  elementary  algebra: 
If  a./b,,  a2/b2, . .  .aa/bn  are  unequal  fraotions  all  of  whose 

denominators  are  of  the  same  sign,  then 

min(a,/bt)  *  Xaj/Zbj  s  maxfa^b,),  (i  »  l,2,...,n).  (9) 

Relation  (9)  leads  to  the  first  of  the  constraints,  the  personnel 
constraint: 

E<b1  a,J.4xi  *  (number  of  stages)  /  (total  number  of  servers).  (10) 

The  TALDTm  constraint  is  given  by  (5) ;  the  transport  constraint  by 
(6) ,  and  the  TCM  constraint  by  (7).  There  is  also  a  constraint 
imposed  by  limitation  upon  the  number  of  vehicles  available  for 
assignment  as  MCT’s. 

6.  A  Numerical  Example.  The  application  led  to  the  following 
linear  programming  problem  (LPP) : 

mi n{l .  354x1  +  0.1648X2  4  2.115X,  +  0.466XJ 


subject  to 

TALDTm:  0.04X.  +  0.Q098X,  +  0.005X,  +0.066X,  2  0.07 
TALDTt :  X.  +  2X,  2  min  TALDTt/P  -  1.24 
TCMS  0.314X.  +  0. 185X2  +  O.llX,  +  0.40x4  2  0.5 
Ko.  personnel:  x1  +  x2  +  x3  +  x4  2  0.36 

Using  STATGRAPHICS,  the  LPP  solution  was  rsached  after  5  pivots. 
The  objective  function  took  the  minimum  value  1.81  hours.  The 
solution  vector  for  the  x,  was 

X-  (0.00000,  0.27556,  0.62000,  0.97272). 

Taking  reciprocals,  these  values  correspond  to  personnel 
assignments  of 

04 »  *11  "  4  (*3 . 6)  ,  S^gf  ■  2  (ei.6)  ,  and  S^a  ■  1, 

which  except  for  the  BMCT  value  are  eminently  reasonable  and  do 
correspond  to  assignments  that  were  actually  made  in  practice.  The 
continuous  nature  of  the  LPP  solution  is  of  course  evident. 
Software  for  an  integer  programming  solution  was  not  available,  but 
might  have  led  to  better  results.  The  impossible  value  for  sip(.T 
preoumably  corresponds  to  the  sensitivity  of  reciprocals  to  small 
values.  Howsvar,  such  an  extreme  value  is  consistent  with  the 
intuition  that  in  the  short-term  AlrLand  Battle,  under  a 
replacement  rather  than  repair  philosophy  ("design  for  discard"), 
the  major  part  of  maintenance  should  be  concentrated  in  mobile 
units  closest  to  where  needed.  Error  estimates  for  this  aspect  of 
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the  problem  remain  open.  There  is  also  ths  matter  of  ths  problem 
bsing  ill-scaled  in  that  tha  coafficiants  of  tho  objactiva  function 
diffar  by  as  much  as  two  ordars  of  magnitude  front  thosa  in  tha 
TALDTm  constraint,  for  instanca.  This  faatura  is  known  on  occasion 
to  make  computer  solution  of  LPP's  impossible. 
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The  Making  and  Use  of  the  "BIG  MAC"  Data  Base 

Fred  M.  Grimes 
John  Riemenschneider 
Combat  Arms  Test  Directorate 
TEXCOM  Combined  Arms  Test  Center 
Fort  Hood,  Texas  76544-5065 


I .  Introduction 

The  ultimate  test  of  new  Army  equipment  is  the  field  test  by 
user  troops.  Engineering  tests  can  measure  certain  aspects  of 
the  equipment's  performance,  however,  it  is  only  in  the  field 
that  the  Army  finds  out  about  the  true  value  of  the  equipment. 

But  getting  meaningful  test  data  from  the  field  is  greatly  more 
difficult  than  getting  data  from  engineering  tests.  Engineering 
tests  can  be  closely  controlled  so  that  there  is  flexibility  in 
collecting  test  data  by  manual  and  instrumented  means.  During 
•field  tests  manual  and  instrumented  data  can  only  be  collected  if 
it  does  not  interfere  with  the  troops  using  the  equipment. 

Further  the  troops  must  be  using  the  equipment  in  a  certified 
operational  manner.  This  constraint  inhibits  collecting  all 
desired  data.  This  constraint  becomes  a  big  time  problem  when 
test  data  is  required  from  force-on-force  engagements  such  as  was 
encountered  in  the  Combat  Vehicle  -  Combat  Performance 
Operational  Assessment  (CV-CPOA)  which  the  Combat  Arms  Test 
Directorate  ( CATD )  was  tasked  to  perform  in  the  spring  of  1987. 


Although  the  mission  was  simple: 

"Gather  empirical  data  on  the  survivability  of  the  Bradley 
Fighting  Vehicle  (BFV)  during  simulated  combined  arms  force-on- 
force  combat  operation  against  a  realistically  configured 
threat" . 

The  execution  was  difficult. 

This  paper  discusses  this  operational  test  and  how  the  vital 
test  data  was  collected  and  reduced  to  a  large  computer  data  base 
which  contained  many  data  elements.  The  computer  data  base  is 
called  the  "BIG  MAC" . 

The  Combat  Arms  Test  Director  who  personally  directed  this 
test  had  the  foresight  to  assign  an  experienced  test  officer  to 
gather  video  and  film  data  of  all  the  events  occurring  during  the 
execution  of  the  test  so  that  a  composite  video  film  could 
summarize  these  events.  A  seventeen  minute  video  resulted  from 
this  effort  and  is  available  for  official  use.  The  video  gives 
an  excellent  overview  of  the  test. 
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Preceding  Page  Blank 


I I .  Background 


The  test  was  specifically  designed  to  collect  data  on  the 
survivability  of  the  currently  configured  BFV  and  three  proposed 
modifications. 

The  CV-CPOA  test  design  methodology  was  driven  by  the 
evaluation  concept  in  the  Operational  Test  and  Evaluation  Agency 
(OTEA)  Independent  Evaluation  Plan.  The  Congressional  defense 
authorization  bill  mandating  the  BFV  assessment  specified  live- 
fire  and  operational  comparison  testing  of  the  Army  version  of  an 
enhanced  survivability  BFV  (M2A1  and  M3A1  HS )  and  the  Department 
of  Defense  version  of  an  enhanced  BFV  (the  ASTB).  The  Army’s 
assessment  of  Congressional  intent  and  implied  taskings  revealed 
the  need  for  a  data  base  that  contained  those  elements  of 
survivability  data  that  would  facilitate  a  comparison  of  proposed 
enhancements  with  a  baseline  using  the  basic  BFV.  An  analysis  of 
all  factors  that  contribute  to  or  degrade  the  survivability  of  a 
combat  vehicle  resulted  in  the  following  key  data  elements  being 
identified  by  CATD  and  OTEA  for  inclusion  in  a  data  base: 

a.  Distribution  of  direct  fire  aiming  points  and  engagement 
angles . 


b.  Exposure  of  target. 

c.  Motion  of  firer  and  target. 

d.  Frequency  of  indirect  fire  engagements. 

e.  Ammunition  and  personnel  on  board  at  time  of  engagement. 

f.  Suppressive  effects  of  firing  port  weapons. 

g.  Target  acquisition  and  engagement  frequencies. 

h.  Distribution  of  engagement  ranges. 

Research  conducted  by  OTEA  and  CATD  revealed  a  dearth  of  such 
survivability  data  on  the  BFV.  Available  data  included  recent 
live-fire  studies  that  had  been  criticized  by  the  Department  of 
Defense,  the  General  Accounting  Office,  and  Congress  as  being 
unrealistic  and  biased.  Other  data  existed  that  was  based  upon 
modeling  analyses  that  did  not  have  credibility  with  the  same 
audience.  Virtually  no  empirical  operational  data  existed.  Even 
though  the  Congressional  bill  directed  only  a  comparison  test  of 
the  two  BFV  versions,  the  absence  of  empirical  survivability  data 
on  the  basic  BFV  led  OTEA  and  CATD  to  design  a  test  which  would 
provide  such  baseline  data.  This  data  had  to  be  the  product  of 
battalion-sized  combined  arms  forces  operating  in  a  realistic, 
simulated  combat,  environment.  The  combat  operations  had  to  be 
representative  of  those  described  in  AirLand  Battle  doctrine,  and 
the  Of’FOR  had  to  be  structured  and  trained  to  represent  the 
prfijff » ed  threat  e  i  ran  1.987-92  in  a  European  .SCORES  V  scenario 
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setting.  Phase  I  trials  were  based  on  this  conoept.  The 
principal  data  output  required  of  phase  I  was  the  actual 
distribution  of  all  direct  fire  engagement  angles  and  aiming 
points . 

In  order  to  respond  specifically  to  Congressional  inquiries, 
direct  comparative  testing  of  the  two  BFV  versions  was  necessary. 
The  optimal  test  design  would  have  had  the  two  enhanced  versions 
of  the  BFV  used  in  the  same  type  operations  with  the  same  size 
forces  and  weapon  types  as  those  in  phase  I.  However,  this  wa» 
infeasible  due  to  nonavailability  of  prototypes  and  surrogates, 
instrumentation  limitations,  and  time  constraints.  Consequently, 
phase  II  was  designed  to  be  a  scaled  down  examination  of  a  Blue 
platoon-sized  force  using  the  vehicles  to  conduct,  offensive 
missions  in  a  simulated  combat  environment.  This  scaled  down 
examination  included  comparison  of  the  basic  BFV,  the  HS  vehicle, 
and  the  ASTB  vehicle,  all  with  500-horsepower  engines,  The  Chief 
of  Staff,  Army,  later  directed  the  inclusion  of  an  ASTB  version 
equipped  with  a  600-horsepower  engine.  Phase  II  trials  were 
based  on  this  conoept.  As  with  phase  1,  the  primary  data 
elements  of  interest  were  engagement  angles  and  aiming  points. 

OTEA's  evaluation  oonoept  was  to  examine  the  comparative 
performance  data  produced  in  the  phase  II  platoon  trials  and 
scale  up  to  the  baseline  data  of  the  phase  I  battalion-sized 
trials.  The  data  from  phases  I  and  11  was  used  as  Inputs  into 
the  US  Army  Materiel  Systems  Analysis  Activity  (AMSAA)  and  the 
Ballistic  Research  Laboratory  survivability  models  to  lend 
further  insights  into  the  BFV  survivability.  The  correlation  of 
the  phase  II  data  with  the  phase  1  baseline  data  and  the  outputs 
of  the  models  and  other  testing  will,  allow  the  Army  to  predict 
the  performance  of  the  various  types  of  BFV' a  with  respect  to 
their  survivability  enhancements. 

Although  Congress  i  one  I.  interest:  focused  on  the  data 
pertaining  to  the  survived! Lity  of  the  BFV  (M2  and  M3),  OTEA, 
anticipating  the  need  for  similar  data  relative  to  other  combat, 
vehicles,  expanded  the  scope  of  the  CV-OPOA  to  include  concurrent, 
collection  of  similar  data  on  the  Ml  tank,  the  improved  TOW 
vehicle  < I TV ) ,  and  the  Ml  13. 

lit.  The  making  of  the  "BIG  MAC" 

The  required  data  was  collected  by  either  instrumented 
systems  or  test  personnel. 

a,  Instrumented  data  and  collection  systems.  Described 
below,  in  general  terms,  are  the*  instrumented  systems  used  to 
simulate  taotlcal  engagements  arid  to  collect  and  record  data. 

(1)  Video  system.  The  video  system  served  as  the 
principal  means  of  data  collection.  Video  tape  was  the  data 
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source  for  aiming  point,  engagement  angle,  target  exposure, 
firer-target  motion,  target  acquisition,  and  player 
identification.  Eaoh  combat  vehicle  and  hand-held  HEAT  weapon 
was  equipped  with  a  coaxially  mounted  video  camera  and  recorder. 
Approximately  two-thirds  of  the  Red  force  vehicles  also  had  a 
through-the-sight  video  camera  and  recorder  installed  which 
reoorded  what  the  gunner  saw.  At  the  beginning  and  end  of  each 
trial,  an  image  of  a  boresight  panel  was  reoorded.  Both  the 
dimensions  and  the  distance  to  the  panel  were  known.  The 
internal  clock  of  the  video  EyBtem  was  synchronized  with  Inter- 
Range  Instrumentation  Group-Format  b  (IRIG-B)  time,  and  the 
rounds  counter  was  set  to  zero.  At  the  end  of  eaoh  trial,  the 
video  tapes  were  removed  and  annotated  with  the  player 
identification  number,  weapons  systems  type,  and  battle  trial 
number . 


(2)  Flash  box  system.  The  flash  box  system  was  used 
with  the  video  system  to  provide  art  additional  souroe  for  the 
data  required  to  determine  the  range  between  the  flrer  and  the 
target.  The  flash  box  system  is  based  on  an  infrared  strobe 
which  flashes  onoe  every  3  seconds.  The  time  of  the  infrared 
flash  is  determined  from  a  computer-driven  oyole  and 
electronically  relayed  from  the  central  computer  to  an  infrared 
strobe  located  on  eaoh  combat  vehicle.  The  flash  of  the  target’s 
infrared  strobe,  which  1h  invisible  to  the  eye,  was  reoorded  by 
video  systems  on  the  firing  vehicles.  The  timing  of  the  Btrobe 
flash  is  a  3-seaond  cycle  and  is  unique  for  eaoh  vehicle.  This 
provides  a  means  of  correlating  the  time  of  the  strobe  flaBh  with 
a  player  identification. 

(3)  Multiple  Integrated  baser  Engagement  System  (MILES) 

(a)  The  need  existed  for  an  Real  Time  Casualty 
Assessment  ( RTCA )  system  to  serve  as  a  tactical  soonario  driver 
by  simulating  direct  fire  weapons  engagements  and  their  effects. 
Emphasis  was  placed  on  causing  crews  and  gunners  to  execute 
proper  gunnery  procedures  under  the  most  realistic  conditions 
possible.  Only  two  RTCA  systems  were  available-~the  TCATA 
Automated  Field  Instrumentation  System  (TAFIS)  and  MILES.  MILES 
was  selected  as  the  principal  component  of  the  RTCA  system 
because  it  met  the  following  critical  test  design  considerations; 

1*  MILES  integrates  the  RTCA  play  of  vehicular  weapons 
systems  arid  individual  soldiers. 

2.  MILES  allows  the  play  of  fully  mobile,  hand-held  HEAT 

weapons . 

3 •  MILES  requires  gunners  to  aim  at  the  center  of  mass 
of  o  target  In  order  to  achieve  a  hit--a  characteristic 
prerequisite  to  collection  of  meaningful  data  on  aiming  point 
distribution . 
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4.  MILES  ia  the  standard  Army-wide  training  simulation 
system  which  is  readiLy  accepted  by  US  (soldiers  and  can  be 
employed  with  minimal  artificiality  and  no  specific  training. 

<b)  The  inherent  limitations  of  MILES,  such  as  fixed 
time  of  flight  for  missiles  and  friendly-only  lethality  codes, 
were  considered  tolerable  since  the  test  design  did  not  require 
the  collection  of  force  exohange  ratios  or  oasuaity  oounts  us 
data  outputs  and  RTCA  system  was  used  only  as  an  exercise  driver. 

(4)  I-M1LES.  To  offset  some  of  the  inherent 
shortcomings  of  MILES  and  to  provide  redundant  data  collection 
capabilities,  an  enhancement  to  MILES  was  procured  specifically 
for  this  test.  I-M1LES  was  used  as  an  addition  to  MILES  and  was 
transparent  to  the  soldier  and  to  the  baaio  MILES.  It  permitted 
programming  of  vulnerability  and  lethality  logic  into  the  host 
MILES,  thus  enabling  the  use  cf  specific  Pk ’ a ,  effective  ranges, 
and  basic  loads  for  all  US  and  threat  weapons  systema. 
Vulnerability  was  varied  by  tactical  mission  exposure.,  For 
example,  Pk  for  the  BFV  fully  exposed  in  the  offense  differed 
from  the  Pk  for  the  BFV  in  hull  defilade  In  the  defense.  I-MILES 
stored  all,  engagement  data  and  provided  a  time-correlated  history 
of  all  MILES  events.  It  also  provided  a  method  of  obtaining  the 
intermediate  data  necessary  to  determine  the  range  between  the 
firer  and  the  target. 

(5)  MILES  Laser  Detector  Decoder  System  (MLDDS).  The 
MLDDS  wrb  used  to  collect  data  on  the  frequency  of  engagement  by 
all  weapons  against  the  BFV  in  specific  quadrants  (front,  right, 
left,  rear).  In  addition  to  the  MILES  sensor  belts  installed  on 
all  combat  vehicles,  separate  MLDDS  sensor  belie  were  installed 
on  the  BFV's  of  one  Blue  force  company  in  phase  I  and  on  the 
IFV’s  and  HS  variants  used  in  phase  II.  When  a  MILES  laser 
struck  a  sensor,  the  laactr  message  and  the  firing  weapon  were 
reoorded  and  a  sequential  history  of  engagements  was  established. 
MLDDS  reoorded  small  arms  engagements .  This  data  was  necessary 
for  assessing  the  effectiveness  of  reactive  tiles. 

(6)  Position  Reporting  and  Recording  System  (FRRS).  The 
PRR8  was  used  to  collect  the  position  location  of  each  combat 
vehicle  at  the  time  of  engagement.  Each  combat,  vehicle  was 
equipped  with  a  mobile  unit  which  transmitted  a  signal  at  a 
prescribed  time  within  eaoh  second.  The  signal  whs  received  by  a 
tower  array  and  relayed  to  PRR3  central  where  computers 
calculated  the  combat  vehicle  location  In  grid  coordinates.  The 
grid  location  and  time  were  recorded  on  magnetic  tape.  Recording 
of  position  location  was  necessary  as  an  Intermediate  step  in 
determining  the  range  between  firer  and  target  for  each  direct 
fire  engagement. 

(7)  Scanning  laser  system.  During  phase  II,  the 
scanning  laser  system  was  used  to  determine  when  line  of  sight 
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existed  between  the  defending  Red  force  vehioles  and  attacking 
Blue  force  BFV's,  In  each  medium  and  long  range  trial,  a 
scanning  laser  was  installed  adjacent  to  each  Red  force  vehicle. 
(For  safety  reasons,  scanning  lasers  were  not  used  in  short  range 
trials.)  The  laser  transmitter  was  mounted  on  a'tripod  in  such  a 
manner  that  the  transmitter  was  at  the  eye  level  of  the  track 
commander  when  the  combat  vehicle  was  in  defilade.  Line  of  sight 
was  established  when  the  transmitter  laser  beam  struck  the  laser 
detector  array  system  installed  on  the  Blue  foroe  vehicle,  The 
laser  signal  identification,  player  identification  of  the  Blue 
force  vehicle,  and  the  time  were  transmitted  by  the  Blue  foroe 
vehicle  to  a  data  collection  system  where  they  were  stored  on  a 
magnetic  tape. 

b.  Manually  collected  data. 

(1)  Prior  to  each  trial  a  player  roster  was  completed 
for  all  participants.  This  form  contained  the  trial 
identification,  unique  player  identification,  and  the 
instrumentation  on  each  player. 

(2)  The  time  each  trial  started  and  ended  and  the  reason 
for  the  ending  were  recorded.  The  type  of  terrain  and  the 
weather  conditions  for  each  trial  and  the  times  that  obsourant 
were  used  were  also  recorded. 

(3)  At  the  beginning  and  end  of  eaoh  trial,  a  controller 
reaorded  odometer  and  engine  hour  readings  for  eaoh  oombat 
vehicle  assigned  to  the  trial,  The  controller  also  recorded  the 
number  of  troopB  on  each  IFV,  CFV,  HS  variant,  and  ASTB  surrogate 
at  the  start  of  the  trial.  During  trial  exeoution,  the 
controller  noted  the  times  when  troops  mounted  and  dismounted. 
Controllers  also  reported  any  administrative  situations  which 
might  have  invalidated  a  trial. 

(4)  Indirect  fire  was  played  only  during  phase  I  trials. 
Manually  collected  data  on  indirect  l'ire  engagements  included  the 
time  of  artillery  fire:  the  type  of  fuse,  ammunition,  and 
weapon;  the  size  and  number  of  volleys;  which  artillery  unit 
should  fire;  size  and  location  of  impact,  area;  and  the 

identif ioation  number  of  the  vehicles  in  the  impact  area. 

(5)  Each  player  completed  a  demographic  questionnaire. 

(6)  At  the  end  of  each  trial,  each  combat  vehicle  turret 
crewman  and  BFV  squad  member  completed  questionnaires  on  the 
perceived  effectiveness  of  their  weapons  to  include  use  of  the 
firing  port  weapons  against  dismounted  troops.  The  Red  force 
dismounted  troops  aloo  completed  a  questionnaire  on  the  perceived 
effectiveness  of  Blue  force  suppressive  fire. 

(7)  At  the  end  of  each  trial,  the  subject  matter  experts 
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evaluated  specific  aspects  of  the  tr ial--maneuver ,  fire  support, 
intelligence,  engineer,  communications,  logistics,  leadership, 
training,  and  use  of  the  combat  vehicle.  They  prepared  a 
narrative  summary  of  the  tactics  and  performance  of  each  force 
for  each  trial.  All  record  trials  were  certified  by  the  senior 
controller  (senior  subject  matter  expert)  as  being  doctrinal ly 
sound . 


(8)  During  phase  II,  three  Blue  force  platoons  rotated 
among  the  HS  variants  and  the  ASTB  surrogates.  Each  crew 
conduoted  six  conseoutive  trials  in  each  vehicle.  At  the 
conclusion  of  the  sixth  trail,  each  crew  member  completed  a 
questionnaire  on  the  differences  in  combat  performance  of  these 
vehioles.  The  Red  force  players  also  completed  a  questionnaire 
on  the  performance  of  the  surrogates. 

Data  collected  by  instrumentation  was  compiled  by  the  test 
support  contractor  and  test  team  and  forwarded  to  the  data 
reduction  center.  Data  collected  manually  was  edited  at  the  teat 
site  and  then  forwarded  to  the  data  reduction  center. 

a.  The  video  tape  and  I -MILES  record  for  a  specific  trial 
were  issued  to  a  data  reducer.  When  through-the-s ight  and 
coaxial  video  tapes  existed  for  a  single  vehiole,  both  tapes  were 
issued  to  two  data  reducers  for  simultaneous  processing.  The 
video  data  reducer  reviewed  the  video  tape  to  determine  when  the 
firing  system  engaged  a  target.  This  time  of  engagement  1b 
referred  to  as  a  trigger-pull.  A  burst  of  10  rounds  from  a  25- 
or  30-millimeter  oannon  constituted  a  single  engagement.  The 
time  of  the  trigger-pull  was  indicated  by  the  rounds  counter 
incrementing  and  by  the  IRIG-B  t.ime  on  the  I-MILES  record. 

b.  The  data  reducer  used  a  specially  fabricated,  three- 

dimensional  scale  model  to  determine  engagement  arigLea  and  aiming 
points  for  the  Ml,  M2,  and  Ml  13  type  vehicles.  The  scale  model 
of  eaoh  oombat  vehicle  is  mounted  on  a  pivot  bo  that,  the  model 
can  be  rotated  through  360°  in  a  horizontal  plane.  The  reducer 
oriented  the  model  so  that  it  matched  l.he  picture  shown  on  the 
video  screen  at  the  time  of  the  trigger-pull.  The*  horizontal 
engagement  angle  between  the  firer  and  the  target  was  read  from 
the  scale  and  entered  on  the  data  reduction  form.  Discrete  ^ 

angles  were  reoorded  for  the  hull  arid  the  turret  orientation.  A 
pivoting  bar  attached  to  the  base  of  the  device  and  oriented  to  a 
vertical  soale  was  used  to  determine  the  vertical  engagement 
angle.  Front,  rear,  Bide,  and  top  view  pictures  of  each  combat 
vehicle  with  4-inoh  grid  squares  sup*  imposed  over  the  pictures 
were  used  to  specify  the  location  of  the  aiming  point.  From 
these  views  the  one  that  matched  the  video  picture  was  selected 
and  the  grid  coordinates  were  recorded. 

c.  The  data  reducer  determined  the  target  vehicle 
identification  by  noting  the  flash  box  counter  number  at  the 
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moment  the  target  vehicle's  strobe  Light  flashed  and  correlating 
that  number  to  the  vehicle  identification  number.  When  the 
pairing  report  from  the  1-MILES  was  available,  it  was  used  to 
validate  target  identification  or  establish  it  when  the  flash  box 
did  not  work. 

d.  Range  between  the  firing  and  target  vehicles  at  the  time 
of  the  trigger-pul.l  was  estimated  by  the  data  reducers  using  a 
stadia  overlay.  The  video  data  reduoer  positioned  the  stadia 
overlay  on  the  video  tape  and  counted  the  number  of  stadia  lines 
for  the  size  of  the  boresight  panel.  This  established  a 
benchmark.  At.  the  time  of  the  trigger-pull,  the  data  reduoer 
placed  the  stadia  source  over  the  video  picture  of  the  target  and 
oounted  the  number  of  stadia  lines  for  the  target  size.  A 
conversion  table  was  used  to  determine  the  range  based  on  the 
number  of  stadia  lines.  This  stadia  range  was  entered  on  the 
data  reduotion  form. 

e.  Range  data  was  calculated  by  a  computer  program  whioh 
used  the  PRRS-developed  position  locations  of  the  firer  and  the 
target  at  the  time  of  the  trigger-pull.  ThlB  method  required 
positive  firer  and  target  identifications  that  were  provided  by 
the  video  tape  and/or  the  I-MILES  printouts. 

f.  If  the  PRRS  and  stadia  rangeB  differed  by  more  than  700 
meters,  a  quality  review  group  examined  both  ranges  to  assess 
their  accuracy,  When  the  PRRS  data  was  questionable  for  either 
the  firing  or  target  vehicle,  like  data  on  other  vehicles  of  the 
same  platoon  was  evaluated  to  verify  the  range.  If  the  PRRS 
range  could  not  be  verified,  the  stadia  range  was  used.  In  those 
cases  where  neither  the  stadia  nor  the  PRRS  range  oould  be 
determined,  senior  data  management  assistants  viewed  the  video 
and  estimated  the  ranges  at  time  of  the  trigger-pull. 

g.  Intervisibility  data  was  derived  from  a  computer  program 
that  correlated  the  positions  of  the  players  at  the  time  a  Blue 
force  vehicle  received  soannirig  laser  energy.  A  computer  program 
calculated  the  duration  of  the  target  vehicle  exposure  and  the 
distance  the  target  vehicle  traveled  while  exposed.  The  computer 
program  provided  an  output  data  set  used  in  the  production  of  the 
phase  II  data  base. 

h.  Data  collection  forms  and  questionnaires  were  edited  by 
quality  assurance  personnel  and  entered  Into  a  computer  data 
file.  The  data  file  was  structured  by  player  identification, 
trial  number,  arid  time  of  data  event. 

IV.  "BIO  MAC"  structure 

The  data  base  is  composed  of  two  distinct  files.  The  primary 
file  (battle  history)  is  an  integrated,  chronological  record  of 
all  the  activities  occurring  during  the  execution  of  a  trial. 


160 


This  is  called  the  "BIG  MAC",  The  other  file  (descriptive  data) 
is  an  integrated  record  of  the  conditions  surrounding  the 
execution  of  the  trial  and  the  subjective  data  collected  during 
the  trial. 

a.  "BIG  MAC"  file.  The  primary  data  file  was  constructed 
during  an  iterative  process  of  data  reduction,  edit,  and  quality 
review . 


(1)  The  initial  format  was  the  file  of  reduced  data  from 
the  1-MILES  and  from  the  video  and  flash  box  systems.  It 
reflected  the  direct  fire  battLe. 

(2)  When  the  initial  file  reached  data  maturity,  a 
computer  program  correlated  the  position  location  of  each  firer- 
target  pair.  This  data  resulted  in  range  being  added  to  the 
file. 


(3)  Concurrent  with  the  range  process,  a  computer 
program  examined  each  engagement  and  calculated  ammunition  usage 
for  each  firer.  The  same  program  also  determined  if  squad 
members  were  on  board  when  the  target  was  a  BFV . 

(4)  When  the  above  steps  were  completed,  the  data  was 
merged  with  the  artillery  data  and  a  time-correlated  file  was 
output  that  contained  data  on  all  direct  arid  indirect  fires. 

(5)  The  integrated  firing  history  was  then  merged  with 
the  data  file  containing  the  trial  start  and  finish  data  on  each 
vehicle.  This  process  continued  with  a  merge  program  that 
incorporated  the  exposure  data  calculated  by  the  Laser  system. 

(6)  The  end  result  was  a  battle  history  file  for  all 

trials. 


b.  Descriptive  data  file.  This  file  contains  the 
information  on  the  conditions  of  each  trial  and  the  subjective 
data  collected.  It  is  organized  as  subordinate  records,  each  of 
which  is  sorted  by  trial  and  time. 

The  mission  of  the  test  team  was  to  produce  a  data  base  on 
operational  survivability  in  a  timely  arid  accurate  manner,  The 
independent  evaluator,  OTEA,  is  responsible  for  data  base 
analysis  and  determining  the  specific  answers  to  questions 
regarding  BFV  survive  hi 1 i ty .  A  copy  of  the  data  base  was 
forwarded  to  OTEA  in  magnetic  tope  format.  The  data  base  was 
derived  from  approximately  6,000  hours  of  video  tape  and  1,000 
collection  forms.  The  data  base  contains  approximately  26,000 
direct  fire  engagements,  1,200  indirect  fire  engagements,  3,000 
intervisibility  segments,  and  ID, 000  descriptive  events.  The 
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total,  data  base  contains  approximately  2  million  data  elements. 
The  data  presented  clearly  demonstrates  the  capability  of  the 
data  base  to  support  analysis  of  the  BFV  survivability  question. 

V.  The  use  of  the  "BIG  MAC" 

The  basic  purpose  of  the  CV-CPOA  operational  test  was  to 
provide  realistic  f oroe-on-foroe  test  data  to  OTEA.  They  would 
use  the  data  in  their  evaluations  to  predict  the  operationally 
survivability  of  the  BFV  and  other  Army  combat  vehicles.  OTEA 
provided  "BIG  MAC"  data  to  the  Ballistic  Research  Laboratory  for 
the  use  in  their  models  to  oalouiate  survivability  results.  OTEA 
completed  its  evaluation  and  reported  its  finding  to  Congress. 

The  findings  led  to  some  design  changes  to  the  BFV.  The  "BIG 
MAC"  had  served  its  purpose  but  the  "BIG  MAC"  is  more  than  a  data 
base  with  a  one  time  use.  The  data  base  is  a  rich  source  of 
force-on-force  test  data.  One  can  play  some  mind  boggling  games 
with  the  number  of  ways  the  data  can  be  sliced  and  diced.  Due  to 
the  large  number  of  different  data  elements,  reports  can  be 
prepared  with  any  number  of  data  seta.  For  example,  the 
following  typo  of  reports  can  be  prepared: 

a.  Direct  fire  weapons  -  Frequency  of'  engagements  and 
engagement  angles 

b.  Direct  fire  weapons  -  Aiming  point  distributions 

c.  Direot  fire  weapons  -  Engagement  range  distributions 

d.  Ammunition  and  personnel,  on  board  BFV  at  time  of 
engagement 

e.  Target  exposure  and  motion  of  flrer  and  target 

f .  Indirect  fire  weapons 

g .  Combat  vehicle  intervisi hility  and  engagements 

h.  Direot  fire  weapons  distribution 

In  February  1988,  the  Department  of  Defense  Office  of  Test 
and  Evaluation  (DOTE)  ashed  that  an  extended  evaluation  of  the 
CV-CPOA  data  be  made  regarding  a  host  of  questions  asked  by  DOTE. 
The  questions  concern  such  things  as,  effect  of  direct  and 
indirect  fire  on  both  the  red  and  blue  target  vehicles,  effects 
of  line  of  sight  on  ranges  cf  engagements,  how  waa  the  ITV 
engaged  on  the  battlefield,  order  effects  on  engagement,  ranges  as 
the  trial  progressed,  etc.  The  Deputy  under  Secretary  of  the 
Army  'Operations  Research)  tasked  OTEA  to  respond  to  these 
questions  and  to  make  the  "BIG  MAC"  available  to  others.  OTEA 
responded  by  forming  a  Study  Advisory  Group  of  Army  organizations 
which  would  have  an  interest  in  the  data.  OTEA  also  obtained 
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contractor  support  to  prepare  a  data  dictionary,  data  base 
documentation  for  the  "BIG  MAC“  arid  also  provide  assistance  in 
responding  to  the  DOTE  questions. 

Several  DAG  members  have  used  data  from  the  "BIG  MAC"  to 
assist  them  in  their  work.  AMRAA  compared  "BIG  MAC"  hit 
distribution  data  with  the  currently  used  cardioid  distribution 
data  For  tanks  and  the  BFV.  The  comparison  showed  that  the  "BIG 
MAC"  data  contained  more  "Head-on"  shots.  As  a  result,  the  Army 
has  decided  to  use  the  "BIG  MAC"  hit  distribution  data  for  future 
AMSAA  modeling  work  for  tanks  and  BFV.  The  Infantry  School  has 
•used  the  "BIG  MAC"  data  in  some  of  their  studies  on  TOW  missile 
firings . 

Upon  completion  of  the  contractor’s  work  on  the  data 
dictionary  and  data  base  documentation,  the  "BIG  MAC"  will  be 
available  to  all  Army  organizations  for  official  use  and  can 
provide  realistic  force-on-force  data  to  assist  them  in  their 
work . 
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Watertown,  Massachusetts  02172-0001 

ABSTRACT 

There  has  been  a  recent  interest  in  determining  high  statistical  reliability 
in  risk  assessment  of  aircraft  components.  This  paper  identifies  the  potential 
consequences  of  incorrectly  assuming  a  particular  statistical  distribution  for 
stress  or  strength  data  used  in  obtaining  the  high  reliability  values.  The 
reliability  is  defined  as  the  probability  of  the  strength  being  greater  than  the 
stress  over  the  range  of  stress  values.  This  method  is  often  referred  to  as  the 
stress-strength  model . 

A  sensitivity  analysis  was  performed  involving  a  comparison  of  reliability 
results  in  order  to  evaluate  the  effects  of  assuming  specific  statistical  dis¬ 
tributions.  Both  known  population  distributions  and  those  that  differed 
slightly  from  the  known  were  considered.  Results  show  substantial  differences 
in  reliability  estimates  even  for  almost  non-detectable  differences  in  the 
assumed  distributions.  These  differences  represent  a  potential  problem  in  using 
the  stress-strength  model  for  high  reliability  computations,  since  in  practice 
it  is  impossible  to  ever  know  the  exact  (population)  distribution. 

An  alternative  reliability  computation  procedure  is  examined  involving 
determination  of  a  lower  bound  on  the  reliability  values  using  extreme  value 
distributions.  This  procedure  reduces  the  possibility  of  obtaining  non¬ 
conservative  reliability  estimates.  Results  indicated  the  method  can  provide 
conservative  bounds  when  computing  high  reliability. 
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INTRODUCTION 


There  has  been  an  interest  in  quantitative  reliability-based  structural  design  for  many 
years.  An  early  example  is  the  structural  reliability  development  by  Freudenthal.1  Stress- 
strength  reliability  computations  are  a  principal  consideration  in  structural  reliability  design. 
Reliability  methods  have  been  considered  for  many  structural  applications  including:  civil  engi¬ 
neering,2  nuclear  reactors,3  fixed  wing  aircraft,4  rotorcraft,5  and  space  vehicle  propulsion  sys¬ 
tems*  Very  high  structural  reliability  is  expected  to  be  achieved  for  most  applications.  A 
reliability  goal  of  0.9(9)  per  flight  hour  was  suggested  in  1955  by  Lundberg4  for  fixed  wing 
civil  aircraft.  Recently,  Lincoln,7’8  using  reasoning  similar  to  that  of  Lundberg,  cited  a  reliabil¬ 
ity  goal  of  0.9(7)  per  flight  for  fixed  wing  military  aircraft.  The  U.S.  Army  has  instituted  a 
new  structural  fatigue  integrity  criterion  for  rotorcraft  which  has  been  interpreted5  as  a 
requirement  for  a  lifetime  reliability  of  0.9(6). 

The  use  of  advanced  materials  whose  structural  properties  are  best  characterized  on  a  sta¬ 
tistical  basis  appears  to  be  a  stimulant  for  increased  interest  in  statistical-based  structural 
design  for  airborne  structures. 

A  significant  feature  associated  with  predictions  of  structural  reliability  is  that  the  conse¬ 
quence  of  a  failure  event  may  be  more  than  reduced  system  performance  or  the  inconve¬ 
nience  of  a  system  being  out  of  service;  structural  failure  can  be  catastrophic  in  terms  of  loss 
of  life  and  property.  In  this  context  it  is  imperative  to  evaluate  the  sensitivity  of  structural 
reliability  predictions  to  uncertainties.  It  appears  that  this  issue  has  received  little  attention 
except  for  a  brief  note  by  Harris  and  Soms9^  and  a  recent  presentation  by  Berens.10 

There  are  many  issues  to  be  faced  in  obtaining  quantitative  structural  reliability  predic¬ 
tions,  Such  issues  include  system  complexity  (many  components,  multiple  failure  modes  in 
each  component,  and  interdependence  of  component  behavior),  sample  or  data  set  size 
associated  with  structural  loading  spectrum  conditions  and  with  mechanical  properties,  and  the 
basis  for  characterizing  structural  qualification  tests  (the  number  of  duplicate  specimens  and 
methods  for  compensation  for  untested  effects  such  as  the  effect  of  environment). 

In  addition,  when  predictions  of  structural  behavior  are  required  in  the  high  reliability 
range,  since  sufficiently  large  data  sets  are  usually  not  available,  it  is  necessary  to  use  parame¬ 
tric  modeling  methods.  Assumed  parametric  functions  permit  extrapolation  from  available  data 
to  determine  the  probability  of  failure.  Since  the  probability  of  failure  is  extremely  small, 
this  will  always  involve  substantial  extrapolation  from  what  can  be  observed  experimentally. 

The  estimated  reliability  will  therefore  depend  strongly  on  the  assumed  parametric  probability 
density  function  (PDF).  Slight  deviation  from  the  assumed  model  in  tail  regions  can  have 
dramatic  effect  on  high  reliability  estimates. 
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In  fact,  one  might  argue,  as  does  Freudenthal,11  that  because  of  the  extrapolation 
involved,  statistically-based  high  reliability  calculations  for  complex  systems  must  always  be 
suspect: 

"When  dealing  with  probabilities  a  clear  distinction  should  be  made  between  conditions 
arising  in  design  of  inexpensive  mass  products  in  which  the  probability  figures  are  derived  by 
statistical  interpretation  of  actual  observations  or  measurements  (since  a  sufficiently  large  num¬ 
ber  of  observations  are  actually  obtainable),  and  conditions  arising  in  design  of  structures  or 
complex  systems.  In  the  latter,  probability  figures  are  used  simply  as  a  scale  or  measure  of 
reliability  that  permits  the  comparison  of  alternative  designs.  The  figures  can  never  be 
checked  by  observations  or  measurement  since  they  are  obtained  by  extrapolations  so  far 
beyond  any  possible  range  of  observation  that  such  extrapolation  can  no  longer  be  based  on 
statistical  arguments  but  could  only  be  justified  by  relevant  physical  reasoning.  Under  these 
conditions  the  absolute  probability  figures  have  no  real  significance  ...." 

Nonparametric  stress-strength  procedures  do  not  require  specific  parametric  assumptions, 
and  so  it  might  be  hoped  that  such  procedures  could  circumvent  this  difficulty.  However, 
Johnson13  has  noted  that  "The  nonparametric  approach  has  one  serious  drawback.  In  return 
for  its  distribution  free  property,  it  is  not  possible  to  establish  high  reliability  even  with  mode¬ 
rate  sample  sizes."  With  respect  to  the  use  of  parametric  models,  Box13  has  observed  "all 
models  are  wrong,  but  some  are  useful,"  meaning  that  no  parametric  statistical  model  should 
be  accepted  uncritically.  Whenever  a  model  is  used,  it  is  the  obligation  of  the  analyst  to 
investigate  the  consequences  of  departures  from  an  assumed  model  which,  though  small,  are 
consistent  with  available  data.  Harris  and  Soms9  has  illustrated  a  "serious  problem  in  the  use 
of  stress-strength  relationships  in  estimating  reliability."  In  particular,  "stress-strength  models 
in  reliability  theory  are  highly  sensitive  to  small  perturbations  in  extreme  tails.”  ’Hie  perturba¬ 
tions  considered  may  arise  from  an  alternative  mode  of  failure  such  as  the  presence  of  a  flaw 
in  a  structure.  Further,  they  note  that  the  problem  cannot  be  eliminated  unless  "astronomi¬ 
cally  large  sample  sizes  are  employed." 

In  the  following,  the  examination  of  the  sensitivity  of  structural  reliability  estimates 
focuses  attention  on  one  of  the  previously  cited  issues:  the  selection  of  a  parametric  PDF. 
The  examination  of  the  sensitivity  of  stress-strength  reliability  estimates  is  extended  to  addi¬ 
tion  perturbation  effects.  The  sensitivity  of  reliability  estimates  to  the  selection  of  parametric 
models  is  considered  with  emphasis  on  graphical  representations.  The  results  are  evaluated 
with  regard  to  the  usefulness  of  parametric  stress-strength  models  for  application  to  the  high 
reliability  regime  of  0.9(6)  to  0.9(7)  when  the  consequence  of  failure  may  be  catastrophic.  An 
alternative  reliability  computation  procedure  is  examined  involving  determination  of  a  lower 
bound  on  reliability  which  can  be  obtained  independently  of  the  assumed  PDFs. 

STRESS-STRENGTH  MODEL 

The  statistical  reliability  as  referred  to  in  this  report  is  determined  in  the  following  man¬ 
ner.  Shown  in  Figure  1  is  the  stress-strength  model  where  f2(s)  and  fi(S)  represent  the 
PDFs  for  the  applied  stress  s  and  material  strength  S. 


11.  rajtU^NTtjAU  A.  Sentiivlty  and  Reliability  of  Mechanical  Systems,  Especially  A  ter  aft  Sauctum.  WADD  Technical  Report 

12.  JOHNSON,  R.  A.  Stress-Strength  Models  for  Reliability,  Handbook  for  Slitlsttca,  Elaevter  Science  Publisher*,  New  York,  v.  7, 1988,  p.  27. 

13.  BOX,  O.  R  P,  Robustness  In  die  Strategy  of  Selenitic  Model  Building.  Robuilnett  In  Sutlitlci,  R.  L  Launer,  and  O.  N.  Wilklnion,  edi, 
Academic  Preai,  Inc,,  New  York,  1979, 
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Normal  stress-strength  model 


Since  the  joint  probability  dR  for  the  strength  being  greater  than  sj  can  be  written  as, 


dR  =  f2  (sj)  d s  /°°  fj  (S)d S 

Sl 

0) 

then  the  reliability  for  all  s  values  is 

R=Cf’«[Cti(S)dS]ds- 

(2) 

PROBABILITY  DENSITY  FUNCTIONS 

A  wide  variety  of  PDFs  may  be  applied  in  obtaining  R  values.  Some  examples  of  PDFs 
are  as  follows: 

The  PDF  most  often  used  in  stress-strength  models  is  the  normal  distribution  (see  Figure  1), 


fN(S)  =  N(S,<72)  =  ^^exp 


(3) 
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where  -  oo  <  S  <  »,  ft  >  0,  and  a  >  0.  The  mean  of  the  population  is  ft,  and  the  stan¬ 
dard  deviation  o  for  this  model. 


A  model  which  is  more  easily  justified  on  physical  grounds  is  the  Weibull  PDF, 


where  S  >  0,  a  >  0,  and  ft  >  0.  Despite  the  relevance  of  the  Weibull  distribution14  to  the 
strength  of  brittle  materials,  it  is  not  often  used,  possibly  because  it  is  more  difficult  com¬ 
putationally  to  obtain  reliability  values  than  with  the  normal  model. 

If  S  follows  the  Weibull  PDF,  then  ln(S)  will  have  an  extreme  value  distribution  with  PDF 


fmln(S)-^exp 


S-ui 


-exp 


The  distribution  of  •  ln(s)  is 


(3) 


‘max 


00-^  exp 


~  (^r)  ~exp  (-(^ir)) 


(6) 


Both  of  the  above  formulas  are  referred  to  as  extreme  value  distributions.  The  use  of 
extreme  value  distributions  in  a  stress-  strength  model  is  illustrated  in  Figure  2.  The  extreme 
value  distribution  parameters  are  related  to  the  Weibull  parameters  as  follows: 


b  m  jj  and  u  ■  -  log  a  . 

In  order  to  obtain  the  population  Weibull  shape  and  scale  parameters  /9  and  a  from  the 
known  population  mean  ft  and  standard  deviation  o,  the  following  approximations  are 
suggested: 


P  *  1.27  ft/o  -0.56 


(7) 


and 

a  m  n/T  lj  . 

The  functions  defined  in  Equations  3,  4,  5,  and  6  clearly  have  different  shapes  and  they 
exhibit  dramatically  different  tail  behavior.  Since  reliability  estimates  depend  strongly  on  the 
extreme  upper  tail  of  the  stress  PDF  and  the  extreme  lower  tail  of  the  strength  PDF,  the 
choice  of  model  will  typically  have  a  substantial  effect  on  the  reliability  estimate.  For  exam¬ 
ple,  R  is  usually  higher  when  calculated  from  the  normal  distribution  than  when  the  extreme 
value  model  is  assumed. 

Applying  PDFs  that  are  capable  of  obtaining  accurate  high  reliability  estimates  (e.g.,  0.9(e)) 
requires  prior  knowledge  of  the  functional  form  of  the  population  PDF  in  addition  to  the 


14.  BURY,  K.  V.  Statistical  Models  In  Applied  Science.  John  Wiley  and  Son*,  Inc.  (New  York,  London,  Sydney,  Toronto),  1973. 
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availability  of  largo  data  sets  (e.g.,  1,000  replicate  specimens).  For  lower  reliability  values 
(e.g.,  0.9),  a  goodness-of-fit  test  for  PDF  identification  with  a  moderate  amount  of  data  is  gen¬ 
erally  adequate.  The  consequence  of  incorrect  PDF  selection  and  limited  sample  sizes  are  dis¬ 
cussed  later  in  this  report. 


Stress-strength:  extreme  value 


Figure  2.  Stress-strength  extreme  value  functions. 


METHODS  FOR  COMPUTING  RELIABILITY  R 

In  determining  R  from  Equation  2  it  should  be  noted  that  the  integration  process  does 
not  determine  an  area.  The  area  A  described  by  the  intersecting  functions  in  Figure  3  does 
not  represent  a  1  -  R  failure  probability.  The  area  A  is  the  probability  (P)  that  either  S  <  T 
or  s  >  T,  that  is, 

A  =  P(S  <  T)  -I-  P(s  >  T)  ,  (8) 

where  T  is  the  point  of  intersection  of  the  two  functions.  The  area  A  is  obviously  not  the 
same  as  the  1  -  R  from  Equation  2  which  determines  P(S  >  s)  jointly  with  P(s). 
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Stress-strength  reliability  model:  normal-normal 


Figure  3.  Stress-strength  Incorrect  unreliability  region. 


Numerical  Integration 

Numerical  integration  procedures  are  usually  suggested  if  a  closed  form  solution  of 
Equation  2  is  not  available.  The  numerical  integration  process  involves  repeated  application 
of  a  method  such  as  Simpson’s  Rule.  The  inner  integral  in  Equation  2  is  evaluated  numeri¬ 
cally  for  each  ordered  Sj  value  i  =  1,  ...,  n  resulting  in  an  Ij(i)  array  of  values.  Each  of 
Il(i)  is  then  multiplied  by  the  corresponding  forming  another  array  l2(i)  =  f2(si)Ii(0-  R 
is  obtained  from  I2  array  by  reapplying  the  numerical  integration  method.  This  process  will 
usually  provide  accurate  results  for  51  <  n  <  101,  where  n  is  the  number  of  mesh  points  in 
the  integration  process.  Simulation  results  showed  that  the  limits  of  integration  can  be 
obtained  from  ±  six  standard  deviations  from  the  mean. 

Closed  form  solutions  are  available  when  the  assumed  stress  and  strength  PDFs  are  both 
normal  or  both  Weibull. 

R  Computation  from  Closed  Form  Solution 

If  both  stress  and  strength  data  can  be  represented  by  normal  PDFs  <?s2)  anc*  N(/*s> 

<7S2),  respectively,  then, 
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R  -  P(S  >  s)  ■  <I> 


(9) 


where  (4>)  is  the  standard  N(0,  1)  normal  cumulative  distribution  function,  and  are 
meant,  and  a2  and  erg2  are  variances  of  the  stress  and  strength,  respectively. 

If  both  fi  and  f2  in  Equation  2  are  Weibull  with  different  scale  parameters  a\  and  a 2, 
but  with  a  common  shape  parameter  ft,  then  the  integration  indicated  in  Equation  2  gives  the 
following  closed  form  expression12 


The  common  shape  parameter  means  that  both  the  stress  and  strength  are  skewed  in  the 
same  way,  which  is  a  serious  limitation.  It  is  much  more  reasonable  to  have  a  stress  distribu- 
tion  with  a  heavy  upper  tail  and  a  strength  distribution  with  a  heavy  lower  tail,  but  this  is 
not  possible  unless  the  shape  parameters  can  be  varied  separately. 

Nonparametrlo  Method 

This  method  does  not  assume  a  PDF  for  either  stress  or  strength  data.  It  determines  reli¬ 
ability  from  the  ordered  array  of  m  stress  (s)  and  n  strength  (S)  values,  where  each  of  the  S 
values  are  compared  with  all  s  values.  R  is  the  proportion  of  times  S  >  s  for  the  total  num¬ 
ber  of  comparisons,  that  is 

R  “  rnn  '*’bcre  “{a?*  $  ■  ("> 

This  method  is  not  useful  for  obtaining  high  reliability  even  for  relatively  large  data  sets. 

It  is  obvious  from  Equation  11  that  for  high  reliability  calculations,  mn  must  be  very  large; 
for  example,  106  would  be  required  in  order  to  obtain  R  of  0.9(«). 

The  Weibull,  normal,  or  other  parametric  PDFs  can  provide  estimates  of  high  R  values 
because  of  their  ability  to  extrapolate  beyond  the  available  empirical  data.  Unfortunately,  the 
amount  of  extrapolation  dependency  determines  the  magnitude  of  relative  error  in  R. 

CONTAMINATED  PROBABILITY  DENSITY  PUNCTION8 

In  order  to  illustrate  the  sensitivity  of  high  reliability  calculations  to  small  deviations  from 
assumed  models,  we  will  take  the  following  approach.  Consider  the  situation  where  with  a 
high  probability  of  1  -  e,  specimens  arc  obtained  from  a  primary  PDF,  white  with  probability  c, 
specimens  come  from  a  secondary  PDF.  This  probability  model  is  referred  to  as  a  contaminated 
model.  The  aecondary  component  is  called  the  contamination,  and  the  probability  e  is  the 
amount  of  contamination. 

An  example  may  help  clarify  this  idea.  Consider  the  situation  where  9796  of  the  time  a 
specimen  is  obtained  from  a  population  of  "good”  specimens  while  the  remaining  3%  of  the 
time  consistently  lower  strength  measurements  are  obtained,  either  due  to  manufacturing 
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defects  or  to  faulty  testing.  The  primary  PDF  would  correspond  to  the  "good"  specimens,  the 
contamination  would  represent  the  distribution  of  flawed  specimens,  and  the  amount  of  con¬ 
tamination  is  £  *  0.03. 


The  following  procedure  is  introduced  in  order  to  examine  the  effects  of  computing  high 
reliability  values  when  uncertainties  exist  in  selecting  the  functions  for  the  stress-strength 
model.  Initially,  high  reliability  values  are  obtained  from  the  normal  stress-strength  model 
(see  Equation  9)  using  known  PDFs  with  different  mean  values  but  equal  coefficients  of  varia¬ 
tion.  The  difference  in  mean  values  were  determined  from  the  required  level  of  high  reliabil¬ 
ity.  Another  R  value  is  then  obtained  by  applying  this  known  distribution  with  a  small 
amount  of  contamination  (fi)  in  order  to  show  an  almost  undetectable  difference  graphically 
between  the  true  and  contaminated  PDFs.  The  effects  of  this  difference  in  the  reliability 
computation  is  discussed  in  the  following  sections  in  order  to  examine  the  sensitivity  of  the 
stress-strength  model  to  the  assumed  PDFs.  This  procedure  provides  an  effective  way  of 
demonstrating  the  effects  of  assuming  a  specific  PDF  in  determining  high  reliability. 

The  normal  PDF  with  variance  contamination  for  the  strength  data  is, 

Nsv  (MS,  <*v2)  -  (1  -«)  N  (ms,  os2)  +  cN  (ms,  Ki  <rS2)  ,  (12) 

where  Ms  and  ers2  are  the  mean  and  variance  for  the  uncontaminated  normal  strength  distribu¬ 
tion,  Ki  is  a  scaling  factor,  and  100  e  is  the  percent  contamination. 

The  strength  distribution  with  location  contamination  is 

Nsl  (Ml,  os  2)  *  (1  -  e) N  (ms,  °s  2)  +  e  N  (ms  ±  K2  os,  os*2)  ,  (13) 

where  K2  is  a  scaling  factor  for  the  mean  MS,  and  the  sign  determines  which  tail  of  the  distri¬ 
bution  is  to  be  contaminated  and  as*2  is  the  variance  on  MS  *  K2?s>  The  location  contami¬ 
nated  PDF  (see  Equation  13)  can  provide  reliability  estimates  to  represent  the  potential  of  a 
secondary  failure  mode.  Contamination  of  the  stress  distribution  would  be  similar  to  that  in 
Equations  12  and  13.  It  was  not  necessary  to  include  contaminated  distributions  for  both 
stress  and  strength  in  order  to  show  substantial  reduction  in  the  high  reliability  estimates. 

The  strength  PDF  contamination  was  sufficient. 

A  linear  relationship  to  obtain  R  for  the  reliability  models  when  a  combination  of  both 
contaminated  and  uncontaminated  stress  and  strength  normal  PDFs  can  be  written  as, 

R  ■  (1  -  Ci)  (1  -  C2)  R  00  +  ci  (1  -  C2)  Rio  +  C2  (1  -  ci)  R01  +  ci  e 2  Rn  (14) 


where  100  £1  and  100  £2  are  percent  contamination  for  the  stress  and  strength  distribution, 
and  the  Ry  values  are  obtained  for  the  case  of  variance  contamination  only;  that  is, 


Rij  - 


MS- Mt 

^To+, 


and  for  location  contamination,  Rkl  would  be 


(15) 
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Equation  14  can  be  extended  to  include  all  combinations  of  variance  and  location  contamina¬ 
tion  simultaneously,  but  it  was  not  necessary  for  this  sensitivity  analysis.  In  Equation  16,  if  i, 
j  *  0,  then  there  is  no  contamination;  for  i,  j  -  1,  then  both  stress  and  strength  are  contami¬ 
nated.  For  example,  if  there  is  contamination  of  variance  of  strength  only,  then 

R  «  (1  “ ®2)  Roo  +  e2  Rot  0?) 


where 


Roi  » 


f  MSp  -  Mt o  \ 

2  .  _  2  J 
'  +  %  ' 


LOWER  RELIABILITY  BOUND 

A  conservative  lower  bound  on  the  reliability  is  introduced  in  order  to  protect  against 
incorrectly  identifying  statistical  functions  in  determining  high  R.  The  bound  is  obtained  from 
a  method  proposed  by  Bolotin,15  and  modified  to  employ  the  extreme  value  PDFs  (see  Equa- 
tions  5  and  6).  The  method  provides  more  conservative  bounds  than  would  be  obtained  from 
standard  methods  which  are  dependent  on  the  assumed  PDFs.  The  selection  of  the  extreme 
value  functions  provides  additional  conservatism  because  of  their  heavier  tails.  The  method  is 
simple  to  use  and  is  not  restricted  to  any  specified  PDF.  The  reliability  bounds  are  (see 
Figure  4), 

i  -  Wj  wa  >  R  >  (i  -  Wi)(i  -  Wa),  08) 

where  (1  -  Wi)(l  -  Wj)  represents,  the  probability  s  <  si  and  S  >  Si,  which  can  be  a  some¬ 
what  conservative  estimate. 

The  lower  bound  is  then, 

Rl  >  (1-Wi)(l-W2)  ,  <19) 

where 

W2  f2 (») d »  and  «  fj  fi  (S) d S 


for  any  choice  of  sj  »  Si 


15.  BOLOTIN,  V,  V.  SuMul  Methods  In  Stntctural  Mechanics,  Hotdcn-Diy,  Ine„  S«n  Fr*ne»ico,  CA,  J.  J.  Bmnd»t*ffer,  ed.,  1969. 
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Stress-strength  extreme  value  model:  Bolotin  R-bound 


Figure  4.  Bolotin  reliability  bounds  using  extreme  value  functions. 


GOODNESS-OF-FIT  TEST 

The  capability  of  determining  desired  PDFs  from  empirical  data  was  investigated.  The 
choice  of  PDF  will  be  shown  in  the  following  sections  to  have  a  substantial  effect  on  high 
reliability  computations,  so  it  is  important  to  examine  model  selection  procedures.  A  statisti¬ 
cal  test1”  of  goodness-of-fit  was  introduced  in  addition  to  graphical  displays  in  order  to  select 
the  desired  PDFs.  Empirical  data  used  in  the  investigation  was  obtained  by  randomly  select¬ 
ing  a  relatively  large  number  of  values  from  a  known  normal  PDF.  A  comparison  of  known 
contaminated  PDFs  and  the  uncontaminated  PDF  is  made  with  respect  to  the  empirical  values. 

RESULTS  AND  DISCUSSIONS 

Variance  Contamination 

Shown  in  Figure  5a  are  reliability  computation  results  and  graphical  display  of  a  normal/normal 
stress-strength  reliability  model,  where  a  1%  (e  =  0.01)  variance  contamination  was  introduced 
and  scaled  by  Kj  =  4.  The  graphical  display  was  obtained  from  application  of  Equations  12 


16.  ANDERSON,  T.  W.,  and  DARLING,  D.  A.  A  Test  of  Goodness-of-fit  J.  Am.  Stalis.  Assoc.,  v.  49,  1954,  p.  765-769. 
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and  13,  where  N(//g,  as2)  is  defined  in  Equation  3.  The  graph  shows  an  almost  undetectable 
difference  between  the  contaminated  and  uncontamiuated  PDFs.  This  indicates  that  the 
choice  of  e  and  K  are  reasonable  with  respect  to  the  potential  differences  between  assumed 
and  actual  PDFs.  However,  the  reliability  values  differ  substantially  (0.9(6)  versus  0.998989). 
This  implies  that  either  one  failure  in  a  million  or  1011  failures  in  a  million  is  predicted 
depending  on  the  selection  of  PDFs  which  can  differ  in  probability  values  by  less  than  0.0005 
in  the  extreme  tail  regions  (see  Figure  5b).  Using  "good”  representative  PDFs  in  the  stress- 
strength  model  in  predicting  only  a  single  failure  will  occur  in  one  million  operations  (e.g., 
number  of  flight  hours)  for  R  *  0.9(6)  can  result  in  a  severe  anticonservative  estimate  since 
for  almost  identical  PDFs,  1011  failures  per  million  could  also  be  predicted. 

The  accuracy  of  the  high  R  estimates  depends  on  the  level  of  precision  in  defining  the 
extreme  tail  of  PDFs.  This  requires  selecting  a  PDF  from  a  data  set  that  accurately  repre¬ 
sents  the  known  population  function  in  the  extreme  tail  regions  with  a  probability  difference 
of  much  less  than  0.0005.  Unfortunately,  this  would  require  an  unrealistically  large  data  set. 

In  current  practice,  if  a  very  large  data  set  is  not  available,  then  PDFs  are  selected  from 
smaller  sets  with  reliance  on  the  functional  representation  in  regions  less  than  first  ordered  or 
greater  than  the  largest  value. 

The  stress-strength  procedure  is  quite  effective  for  the  range  of  R  values  between  0.5 
and  0.95  since  usually  in  the  extrapolation  process,  a  small  difference  in  the  extreme  tail  prob¬ 
abilities  values  will  not  effect  the  required  accuracy  in  R.  Reliability  results  from  uncontami¬ 
nated  and  variance  contaminated  (e  «  0.05  and  Kj  ■  5)  PDFs  showed  no  differences  for  a 
known  R  ■  0.95.  Unfortunately,  in  order  to  obtain  high  reliability,  extrapolation  into  the 
extreme  tail  of  the  PDFs  is  required,  thereby  increasing  the  required  level  of  precision  neces¬ 
sary  to  distinguish  between,  for  example,  0.998  and  0.9(6). 

In  order  to  demonstrate  the  uncertainties  in  selecting  specific  PDFs  from  empirical  data 
when  computing  high  reliability  values,  the  following  displays  are  shown  in  Figure  6.  In 
Figure  6a,  a  plot  is  shown  of  the  empirical  normal  cumulative  density  function  (CDF)  and  the 
corresponding  contaminated  and  uncontaminated  normal  distribution  functions  where  the  mean 
is  50  and  standard  deviation  (SD)  is  5,  with  sample  size  ft  -  100.  Reliability  values  are  also 
tabulated  from  the  stress-strength  model  results  using  all  six  candidate  functions.  For  exam¬ 
ple,  R(3,  5)  is  the  reliability  obtained  from  variance  contamination  of  3%  and  a  scale  of  5 
for  variance.  A  statistical  goodness-of-fit  test16  that  measures  the  relative  differences  in  the 
tail  region  of  the  distributions  was  applied  in  addition  to  visual  inspection  in  order  to  estab¬ 
lish  if  each  function  could  represent  the  CDF  of  the  ranked  data.  Results  showed  this  to  be 
true;  see  Figure  6b  for  the  tabulated  observed  significance  level  (OSL)  which  show  in  all 
cases  OSL  >  0.05,  a  requirement  for  the  assumed  function  to  be  considered  from  the  same 
population  as  the  empirical  data. 

The  results  show  that  although  each  distribution  fits  the  data  quite  well  (see  Figure  6b), 
there  is  a  large  relative  difference  in  R  values:  0.9(6)  for  R(U.C)  and  0.9957  for  R(3,  5). 

In  Figure  7,  the  results  are  similar  to  those  in  Figure  5.  The  variance  contamination  was  \% 
with  a  scale  factor  of  6  for  both  erg  and  o%.  Again,  although  the  functions  are  similar,  the  rela¬ 
tive  reliabilities  differ  substantially  (0.9(6)  versus  0.9977197).  As  was  the  case  in  Figure  5, 
severe  consequences  could  exist  if  R  «  0-9(6)  >*  assumed  and  the  actual  reliability  was 
0.9977197.  Since  this  could  result  in  a  number  of  premature  failures,  2280  in  one  million, 
occurring  compared  to  the  assumed  one  failure  in  a  million.  The  results  showed  a  low  level 
of  sensitivity  to  the  selection  of  the  factor  K\. 
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Normal  stress-strength  models 


Figure  5a.  Stress-strength  normal  functions  with  and  without  variance  contamination. 


Normal  stress-strength  models  with  contamination 


Figure  5b,  Detailed  region  of  intersecting  functions  (see  Figure  5a). 
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Normal  stress-strength  models 


Figure  7.  Rellabllity/normal  functions  with  and  without  variance  contamination. 


Location  Contamination 

In  Figure  8,  reliability  computation  results  and  a  graphical  display  of  the  stress-strength 
models  are  shown.  The  contaminated  functions  were  obtained  from  1%  (e  =  0.01)  location 
contamination  as  defined  in  Equation  13  where  K2  =  4  and  the  (-)  value  is  used  for  strength 
and  (+)  value  for  stress.  The  contamination  in  this  case  represents  a  secondary  failure  mode 
not  considered  when  assuming  a  specified  function  from  the  test  results.  For  example,  ignor¬ 
ing  the  possibility  that  one  in  every  100  parts  may  have  a  lower  strength  level,  say  4  standard 
deviations  from  the  mean,  can  result  in  the  reliabilities  tabulated  in  the  figure.  That  is,  for 
the  assumed  correct  model,  R  =  0.9(6),  and  the  actual  case  where  there  was  a  lower  strength 
level  having  one  chance  in  100  of  occurring  resulted  in  R  =  0.999459.  Figure  9  provides 
similar  results  to  those  in  Figure  8  except  there  is  a  greater  difference  in  reliability  values 
0.9(6)  versus  0.991012  due  to  a  greater  shift  (K2  =  6)  in  the  mean  value  for  the  contami¬ 
nated  PDF.  With  a  1%  contamination  this  result  is  predictable  since  one  in  a  hundred  times 
a  failure  should  occur  because  /*s  *  K20  ,s  less  than  the  mean  of  stress  value.  The  above 
figure  shows  the  consequences  of  not  being  able  to  identify  the  correct  function  because  of 
the  inability  to  always  detect  a  flawed  component.  The  result  is  the  determination  of  an 
overly  optimistic  reliability  value  when  the  true  reliability  could  actually  be  orders  of 
magnitude  less. 


180 


sro  oro  vro  tro  sio  oio  soo 


Normal  stress-strength  models 


Figure  8.  Reliability/normal  functions  with  and  without  location  contamination. 


Norma?  stress-strength  models 
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Figure  9.  Reliability/normal  functions  with  and  without  location  contamination. 
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The  results  in  Figure  10  arc  similar  to  those  in  Figure  5  except  these  were  obtained  from 
E  »  0.03  and  K\  =  3.  If  the  estimated  R  =  0.9987350  is  obtained  from  the  empirical  data 
and  a  higher  R  value  is  required  (R  =  0.9(6)),  then  a  material  with  either  greater  strength  or 
less  contamination  would  be  required.  In  order  to  obtain  the  required  0.9(6)  from  the  origi¬ 
nal  contaminated  model,  a  mean  strength  of  87  is  required  (see  Figure  11).  The  mean  of  87 
requirement  may  not  be  acceptable  to  the  designer,  but  this  situation  can  occur  if  there  is  a 
substantia]  amount  of  dispersion  in  the  strength  data  resulting  in  a  long-tailed  PDF.  The 
above  situation  shows  when  a  potentially  over-design  situation  could  occur  because  of  the 
inability  to  identify  the  correct  PDF  in  the  stress-strength  model  due  to  inherent  sensitivity 
and  lack  of  information  in  the  tail  regions.  This  could  prevent  a  good  design  from  being 
accepted  if  it  is  required  that  the  assessment  of  the  design  be  based  upon  reliability  only. 


10  20  30  40  50  60  70 

Stress  or  Strength 

Figure  10.  Reliability/stress-strength  with  and  without  contamination. 
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Normal  stress-strength  models 


Stress  or  Strength 


Figure  1 1 .  Increased  strength  requirements  for  high  reliability. 


The  display  in  Figure  12  presents  four  possible  reliability  values  for  the  case  where  the 
means  and  standard  deviation  are:  stress  (24,  2.4)  and  strength  (51,  5.1).  The  result  from 
the  uncontaminated  normal  is  0.9(6).  R  =  0.995043  was  obtained  from  the  contaminated  PDF 
application.  Since,  as  was  shown  previously,  the  stress-strength  model  will  often  provide 
either  relatively  very  high  or  low  R  values  depending  upon  the  chance  selection  of  the  PDFs. 
In  order  to  compensate  for  the  uncertainty  in  selecting  the  PDFs  for  stress  and  strength  data, 
extreme  value  distributions  are  introduced  (see  Figure  2)  in  the  reliability  computation.  This 
resulted  in  R  =  0.999045.  Unfortunately,  this  did  not  provide  a  value  lower  than  the  contami¬ 
nated  model  result  of  0.9950428.  In  order  to  obtain  additional  conservatism  in  the  R  esti¬ 
mate,  a  modification  of  a  method  by  Bolotin  is  examined  involving  the  determination  of  lower 
bound  on  R  (see  Figure  4  and  Equation  19)  in  conjunction  with  the  extreme  value  PDFs. 

The  resultant  lower  bound  estimate  of  0.9796063  provides  a  significantly  lower  value  than  that 
of  the  contaminated  model.  This  was  also  true  for  all  contaminated  models  in  this  study. 

This  lower  bound  estimate  could  provide  some  security  in  estimating  R,  although  results 
may  be  excessively  conservative  for  some  practical  applications.  In  Table  1,  the  distribution 
of  R  values  as  a  function  of  the  sample  size  is  presented.  R  values  were  obtained  from 
repeated  application  of  the  uncontaminated  stress-strength  model  of  Figure  5  using  randomly 
selected,  normally  distributed  samples.  For  a  sample  size  of  10,  R  ranges  from  0.9(6)  t0 
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0,998417  indicating  the  instability  associated  with  small  samples.  Higher  order  quantiles  (e.g., 
60%)  were  not  included  since  they  were  all  greater  than  0.9(6). 

Table  1.  DISTRIBUTION  OF  R  VERSUS  SAMPLE  SIZE 


Reliability  R 


Distribution 

(%) 

Sample  Size 

10 

50 

100 

1000 

0.1 

0.998417 

0.999932 

0.999980 

0.999998 

(1583) 

(68) 

(20) 

(2) 

1 

0.999160 

0.999968 

0.999991 

0.999998 

(840) 

(32) 

(9) 

(2) 

10 

0.999943 

0.999994 

ft  (vywtQ 

0.999999 

(57) 

(6) 

(2) 

(D 

25 

0.999994 

0.999998 

ft  tyyyyyt 

0.999999 

(6) 

(2) 

(i) 

(1) 

50 

0.999999 

0.999999 

ft QQQQQQ 

0.999999 

(D 

0) 

(1) 

(D 

( )  Corresponding  Number  of  Failures  Per  Million 


Extreme  value  stress-strength  model 
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A  sample  size  of  50  or  100  provides  reasonable  stability,  and  a  sample  of  1000  shows 
essentially  no  variability.  The  results  from  Table  1  show  that  for  a  sample  of  1000,  an  esti¬ 
mate  of  R  ■  0.9(6)  would  be  acceptable.  This  is  not  necessarily  correct  since  results  from 
the  table  only  address  the  sample  size  issue  which  is  independent  of  the_uncertainties  in  the 
PDFs  selection  process.  There  are  two  requirements  for  obtaining  accurate  high  reliability  val¬ 
ues  from  the  strength  model:  large  samples  (n  >  1000)  and  knowledge  of  the  population 
PDF.  Reliability  estimates  of  0.95  are  much  less  sensitive  to  the  PDF  assumption.  If  there 
is  a  secondary  failure  mode  due  to  occasional  undetected  poor  manufacturing  of  the  material 
or  an  unusually  large  load  occurs  that  is  not  accounted  for  in  the  design  process,  then 
unknown  lower  reliability  values  (R  <  0.95)  can  exist. 

CONCLUSIONS 

High  reliability  estimates  from  application  of  the  statistical  stress-strength  model  can  vary 
substantially  even  for  almost  undetectable  differences  in  the  assumed  stress  and  strength 
PDFs.  Specifying  high  R  values  (e.g.,  0.9(6))  f°r  acceptable  structural  design  can  result  in 
higher  failure  rates  than  anticipated  if  the  assumed  PDFs  contain  shorter  tails  than  actually 
exist.  Over-design  situations  can  also  occur  when  excessively  long-tailed  PDFs  are  applied  to 
the  stress-strength  model.  An  effective  method  for  identifying  this  nonrobust  behavior 
involved  application  of  contaminated  and  uncontaminated  PDFs  in  the  determination  of 
reliability  values. 

A  suggested  method  for  obtaining  a  lowc  bound  on  the  reliability  estimate  provided 
potentially  overly  conservative  results  but  was  effective  in  determing  values  that  were  lower 
than  any  of  the  R  values  computed  for  the  contaminated  models. 

The  authors’  position  regarding  the  computation  of  high  reliability  of  0.9(6)  agrees  with 
Breiman17  who  says  "The  probability  of  failure  Pf  ■  1  x  10*6  has  an  Alice  in  wonderland 
flavor  and  should  be  banned  from  nonfiction  literature."  It  is  therefore  recommended  that  if 
high  reliability  calculations  are  absolutely  essential,  then  the  results  should  be  subjected  to  a 
sensitivity  analysis  using  contaminated  distributions.  High  reliability  values  are  meaningful  only 
when  these  values  are  not  substantially  affected  by  an  amount  of  contamination  (e)  consistent 
with  the  sample  sizes,  and  a  severity  of  contamination  which  is  identified  by  engineering 
judgement. 
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IMPROVING  NONPARAMETRIC  TOLERANCE  LIMITS  FROM  POOLED  DATA 
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U.S.  Army  Materials  Technology  Laboratory 
Watertown,  Massachusetts  02172-0001 

Abstract 

This  paper  introduces  a  method  for  obtaining  an  improved  tolerance  limit 
value  for  a  small  sample  S  of  material  strength  data  by  pooling  with  a  much 
larger  sample  L.  This  value  represents  a  ' B '  basis  material  property  number 
defined  as  the  95%  lower  confidence  bound  on  the  tenth  percentile  of  the  popula¬ 
tion  distribution. 

In  the  pooling  process  both  data  sets  are  transformed  to  a  common  mean  value 
of  zero  in  order  to  pool  samples  with  significantly  different  strength  levels. 
Equality  of  variance  is  required  between  S  and  L.  The  basis  values  are  obtained 
from  the  pooled  samples  by  application  of  both  nonparametric  and  parametric 
statistical  models.  Monte  Carlo  studies  showed  that  by  pooling  both  data  sets  S 
and  L,  values  could  be  obtained  that  were  less  conservative  with  lower  variabil¬ 
ity  than  from  application  of  S  alone. 


Preceding  Page  Blank 


Introduction 

The  statistically  based  material  property  value,  or  B-basIs  value,  Is  a 
statistic  which  Is  less  than  the  tenth  percentile  of  the  population  with  proba¬ 
bility  .95.  That  Is,  the  B-basIs  value  Is  a  95*  lower  tolerance  limit  for  the 
tenth  percentile.  In  Figures  la  and  lb,  a  graphical  display  Is  shown  for  the 
basis  value  probability  density  function  for  sample  sizes  of  n  ■  10  and  50  from 

4  U 

a  standard  normal  population.  The  dotted  vertical  lines  Indicate  the  10 
percentile  (x  10)  of  the  population  and  the  probability  (basis  value 
<  x  10)  ■  .95  for  the  basis  value  probability  density  function.  The  graphical 
display  of  the  basis  value  density  functions  show  much  less  dispersion  for 
n  ■  50  than  for  n  ■  10.  Therefore,  small  samples  will  usually  result  In  lower 
basis  values.  In  3*  ^  various  procedures  are  described  for  determining 

the  statistical  property  values. 

The  motivation  for  the  work  described  In  this  paper  resulted  from  a  need  by 
the  aircraft  Industry  to  obtain  a  less  conservative  statistically  based  material 
property  value  from  a  small  sample  of  composite  material  strength  data.  Here 
'conservative'  Is  to  be  Interpreted  to  mean  'excessively  low',  which  corresponds 
to  a  design  engineer's  use  of  the  word.  Statistical  conservatism,  that  Is  a 
confidence  exceeding  the  nominal  level  of  .95,  need  not  be  present  for  'engi¬ 
neering  conservatism'  to  be  a  problem. 

The  use  of  small  samples  reduces  the  amount  of  testing  and  consequently  the 
Manufacturing  cost  of  composite  aircraft  structures.  For  example,  In  order  to 
qualify  a  composite  material  to  be  used  In  the  manufacture  of  a  commercial 
aircraft,  the  FAA^  requires  property  values  for  tension,  compression,  and 
shear  tests  subjected  to  the  environmental  conditions:  hot-wet,  cold-dry,  and 
room  temperature  for  three  separate  batches  of  material.  In  the  development  of 
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a  composite  tall  section  by  one  of  the  major  aircraft  companies  the  cost  of 
testing  Mas  more  than  20  million  dollars.  In  addition  to  the  cost,  exces¬ 
sively  conservative  basis  values  can  also  result  in  an  overdesign  situation, 
since  the  value  often  provides  Information  In  determining  structural  design 
allowables. 

In  order  to  avoid  the  penalty  associated  with  using  small  samples  In  the 
tolerance  limit  computation,  a  procedure  Is  Introduced  In  this  paper  Involving 
pooling  a  large  sample  with  a  smaller  one  In  order  to  obtain  the  property  value. 
This  Is  done  In  order  to  reduce  the  Inherent  variability  that  occurs  from  apply¬ 
ing  the  smaller  data  set. 

In  the  pooling  process  the  larger  data  set  should  be  obtained  from  prior 
available  test  results  or  from  less  expensive  tests.  Ideally,  both  samples 
should  be  from  the  same  test  and  environmental  conditioning  process.  Various 
combinations  of  tests  and  ply-orientations  are  shown  In  Figure  2  showing  the 
sources  for  combining  the  data  sets.  In  the  pooling  process  It  Is  assumed  for  a 
given  material  (eg.,  graphite-epoxy)  there  are  similar  failure  modes  and  conse¬ 
quently  equality  of  variance^  between  the  samples. 

In  order  to  avoid  the  uncertainties  Involved  In  Identifying  a  statistical 
model  from  a  small  sample  when  computing  the  basis  value,  two  nonparametrlc 
methods  (Ferguson^  and  the  Modified  Hanson-Koopmans^)  are  Introduced.  In 
applying  [7],  the  larger  set  represents  the  prior  and  the  smaller  one  the  empir¬ 
ical  data.  In  [8]  an  ordered  array  of  strength  measurements  are  obtained  from 
the  pooled  data  sets.  The  tolerance  limit  Is  determined  from  a  specific  ordered 
value  multiplied  by  a  factor  determined  from  the  sample  size  of  the  pooled  data. 
A  parametric  method^  was  also  applied  In  the  study  In  order  to  compare  the 
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effectiveness  of  the  various  methods.  This  Involved  a  Wei  bull  analysis,  where 
the  shape  parameter  Is  obtained  from  the  pooled  sample. 

A  Reduced  Ratio  method^10^  Is  recommended  by  the  MIL-5  handbook  for  obtain¬ 
ing  basis  values  from  small  samples.  The  procedure  involves  determining  the 
ratios  of  paired  observations  from  a  prior  data  set  with  a  known  basis  value  and 
a  new  data  set  for  which  the  value  is  to  be  determined.  A  lower  confidence 
bound  Is  determined  for  the  ratio  values.  The  basis  value  Is  then  obtained  from 
multiplying  the  bound  by  the  prior  value.  This  method  Is  often  overly  noncon¬ 
servative. 


Determination  of  Basis  Values  -  Nonparametrlc  Bayesian  Method 
The  nonparametrlc  Bayesian  basis  value  Is  obtained  from  the  following.  Let 
[xljl°  rePre$ent  current  empirical  data  which  the  basis  value  Is  to  repre¬ 
sent  and^tj^m  the  prior  data  obtained  from  test  results  or  an  assumed  cumula¬ 
tive  distribution  function  (CDF)  F(t). 

Posterior  Distribution 

The  posterior  distribution  using  Bayes  rule  can  be  written  as 

Fn(tfxl'  x2'  •••  V  tj)+  ^ 

+•  A\ 

where  n  Is  the  sample  size  of  the  current  empirical  data  set  and  *<(R)  Is  the 
prior  data  sample  size.  $x((-  «•«»  t])  represents  the  number  of  values  from  the 
prior  distribution  less  than  or  equal  to  t  which  are  determined  from  the  equal¬ 
ity  between  the  corresponding  ordered  value  of  the  CDF's  and  the  quantile  of  t. 
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$x  ((-  t])  represents  the  number  of  current  empirical  data  values  x  less 

than  or  equal  to  t  determined  from  the  equality  between  their  corresponding 
combined  ordered  values  of  the  CDF's  of  x  and  t  and  the  quantile  of  t  where 
represents  the  measure  giving  the  mass  of  one  to  the  point  x. 

Equation  (1)  can  be  rewritten  In  the  following  manner,  if 

F0(t)  -  c*((-  ,  t])/<V(R),  (2) 

represents  the  prior  CDF  and 

Fn(t  xj»  x2*  •••  "  A  **  *  (3) 

represents  the  current  empirical  data  CDF,  then 

Fn(t  xj*  x2»  '*•  xn)  "  F^(t)  ♦  (1  "  F^)F^(t|xj,  Xg,  •••  x^)  (4) 

represents  the  posterior  shown  as  the  weighted  ordered  array  of  CDF's,  where 


P 


(5) 


determines  the  weight  for  the  prior  and  current  data.  By  Inverting  the 
CDF's  In  Equation  (4),  a  value  of  t  can  be  obtained  corresponding  to  a  specified 
quantile. 


Illustrations 

A  few  simple  examples  will  show  the  relative  ease  In  obtaining  the  posterior 

A 

distribution  F  . 

n 


Let 

t  ■  1*  2,  3,  4,  5*  and  FQ  ■  *2,  ,4,  .6, 

0h 

O 

• 

m. 

GO 

• 

x  *  1)  2,  3,  4|  5)  and  F^  *  >2|  *4|  «6| 

.8,  1.0 

Since 

Fn(t  Xj,  x2,  ...  xn)  "  Pn  FQ  +  (1  -  Pn)Fn 

Then 

Fn(l  1,  2,  3.  4,  5)  -  (.5)(.2)  +  (.5)(.2)  - 

.2 
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This  is  the  estimate  for  first  ordered  value  for  x  obtained  from  a  good  choice 
for  the  prior  F0. 

If  t  ■  1,  2,  3,  4,  5  and  F0  ■  ,2,  .4,  .6,  .8,  1.0, 

x  ■  6,  7,  8,  9,  10  and  Fn  ■  .2,  .4,  .6,  .8,  1.0 

Then  1n<6  6,  7,  8,  9,  10)  -  (1.0)(.5)  +  (.2)(.5)  -  .6 
This  Is  a  poor  estimate  since  ,2  Is  the  correct  result.  An  unacceptable  prior 
produced  this  result*  The  CDF  value  for  t  ■  1  using  the  above  data  Is 

Fn(l  6.  7,  8,  9,  10)  •  ( ,2)(*5)  •  .10 
A  Bayes  estimate  of  the  mean  can  be  obtained  as  follows, 

"A0  I*  the  mean  determined  from  a  prior  data  set  and  xn  Is. the  mean  of 
current  sample  of  test  results,  then  the  Bayes  mean  value  for  x.  Is 

A  n 

X(n(xi.  x2,  ...  xn)  ■  ?nA0  4  (1  -  PnJV  (6) 

where  Pn  Is  defined  In  Equation  (5). 

Nonparamstrlc  Tolerance  Limit  on  the  Bayesian  Quantile  Estimate 
Since  a  basis  value  as  described  previously  requires  a  tolerance  limit  on 
the  quantile  estimates,  the  following  describes  the  process  for  obtaining  that 
limit.  Initially,  a  random  sample  F(y)  of  size  M  ■  t<  (R)  ♦  n  Is  assumed  Inde¬ 
pendent  of  the  mixture  of  the  prior  and  empirical  data  sets  shown  In  Equation 
(4).  By  ordering  the  sample  F(y),  the  following  Y(1),  Y(2),  ...  Y<M)  values  are 
obtained.  The  probability  density  function  for  Y^j,  1  <  1  <  M  can  be  written 

r(uM)r(( 


O-a) 


f  -  u.)  m) 


(7) 
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where  z^j  ■  F(Y^j)  and  1  ■  uM  with  u  representing  the  CDF  value  corresponding 
to  1.  The  tolerance  limit  y*  for  Is  defined  as, 

P(yq  >  y*)  ■  »  -<X‘  P[F(yq)  >  F(y*)] 
where  yq  is  the  lOOq  percentile  of  y.  Since 


F(y  £yq)  ■ 


I 


*  rtM)^"-1  (i -a) 


(I  -U^M  —  I 


r(KM)r((  i-  u.)  m) 


(8) 


from  Equation  (7),  then  a  1  -  **  tolerance  limit  on  yq  can  be  obtained  from  the 
following  solving  for  u; 


1 


% 

f(z)  dz  ■  1  -  o< 


(9) 


where  f(z)  denotes  the  Integrand  In  Equation  (6). 

In  the  case  of  the  'B*  basis  computation,  c<  ■  .05  and  q  ■  ,10.  Equation 
(9)  can  then  be  written  as. 


JO 


r<M)x 


UM-I 


O-a) 


(1-UjM-l 


r(uM)r((f-u.)M) 


r  (10) 


See  Table  I  for  tabulation  of  u  and  M  values  that  satisfy  Equation  (10). 
If,  for  example,  uM  ■  1  and  u  ■  ,034,  then 


(&t) 


(11) 


and  M  ■  29,  that  Is  P(Y^0  _>  Y^j)  >  ,95,  The  tolerance  limit  Is  then  the 
first  ordered  value  Y^j  for  the  sample  size  of  M  ■  29.  A  'B'  basis  value  can 
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now  be  obtained  from  the  CDF  In  Equation  (4)  by  the  following  procedures. 

Solving  for  u  In  Equation  (10)  determines  the  lower  tolerlnce  limit  of  the  CDF 
of  sample  size  M  and  from  some  distribution  F(y)  where  the  1  ■  uM. 

Obtaining  a  lower  ordered  CDF  value  from  Equation  (4)  that  Is  approximately 
equal  to  u  determines  the  1  -  of  tolerance  limit  of  the  p^  quantile  of  the 
posterior  CDF  for  a  sample  size  M. 

Example; 

If  there  Is  only  prior  data  [t^30  and  a  'B'  basis  value  Is  required  where ... 

tj  ■  5,  6,  7,  8,  12.  16,  20,  25,  ...  40 

and  F^(t)  **  .033,  .066,  .099,  ...  1.0 

then  for  M  ■  30,  u  ■  .034  from  Table  I.  The  basis  value  t.  Is  determined 

l  3 

from  an  approximate  solution  of  u  F (t j )  resulting  In  tj  ■  5;  therefore,  the 

first  ordered  value  of  the  prior  represents  the  ' B V  basis  value,  which  Is  the 
same  as  the  nonparametrlc  quantile  sign  test^-11^  result,  when  the  sample  size  Is 
30. 


Basis  Property  Values  -  Nonparametrlc  (  <HK)  Process 
A  nonparametrlc  procedure  (HK)^  for  estimating  tolerance  limits  Is  Intro¬ 
duced  for  computing  the  ' B '  basis  value  for  any  sample  size  _>  2.  The  method  Is 
a  modification  of  Hanson-Koopmans^12^  process.  The  modification  has  reduced  the 
excessive  conservatism  In  computing  property  values  when  compared  with  the 
original  HK  method. 

The  method  Involves  the  following.  Let  Xj,  ...  xn  be  the  order  statistic  of 
an  Independent  and  Identically  distributed  sample  from  a  continuous  distribution 
F.  Assume  that  F  Is  log-convex,  that  Is  -  log  F(X)  Is  a  convex  function.  The 
class  of  log-convex  functions  Includes  a  large  enough  group  of  distributions  so 
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that  following  procedure  Involving  log-convex  functions  can  be  considered  non- 
parametrlc. 

The  lower  tolerance  limits  of  the  form 

Trs  ■  *  xr  +  (1  -  (12) 

can  be  obtained  where  r  <  s  and  K  _>  1.  When  Tps  Is  used  with  positive  data 
values,  negative  tolerance  limit  can  be  obtained  which  are  not  valid  If  the 
distribution  F  Is  zero  for  any  negative  values.  A  practical  solution  to  this 
problem  1$  to  apply  the  Hanson-Koopmans  approach  by  taking  the  log  of  the  data 
x.  that  Is, 

T*  ■  K  log  Xr  +  (1  -  K)log  *,  (13) 

and  then  obtain  by  exponentiation  the  following 

?  .  1«9  +  e(l  .  K)lCg  x,  •  Xj  (  (») 

For  most  distributions  of  Interest,  still  provides  conservative  tolerance 
limits,  although  technically  T^,  Is  valid  for  a  class  of  distributions  smaller 
than  the  log-convex  class  corresponding  to  T^. 

In  order  to  determine  the  'B'  basis  value,  the  1,  j,  and  K  values  are 
obtained  for  a  given  n  are  shown  In  Table  II. 

Wei  bull  Model  Property  Values  -  Pooling  Data  For  Shape  Estimate 
A  Welbull  method  was  Introduced  by^  In  order  to  obtain  ' B '  basis  material 
property  values  for  various  tests  and  conditioning  processes  of  composite  mate¬ 
rials.  Although  the  Welbull  scale  parameters  can  differ  depending  on  the  type 
of  test  or  condition,  the  within  sample  variances  are  usually  similar.  By  this 
assumption,  the  data  from  the  various  sources  are  pooled  In  order  to  determine 
an  approximate  population  shape  parameter  c*  *  for  the  Welbull  distribution.  In 
order  to  obtain  a  basis  value  for  some  specified  smaller  data  set  Y  (usually 
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from  tests  and  conditioning  that  requires  considerable  expense),  the  following 


relation  Is  applied; 

‘B1  »/9y[*  ln(0.90)31/o4*C2n/ (15) 
where  the  quantity  9g  Is  the  .95  Chi-square  quantile  with  2n  degrees  of 
freedom,  and  n  Is  the  number  of  Y  values,  /$y  Is  the  Welbull  scale  parameter 
estimated  from  the  smaller  sample  of  size  n.  Equation  (15)  Is  not  valid  unless 
can  be  adequately  represented  by  a  large  enough  sample  to  provide  a  good 
approximation  to  the  true  o<  value.  The  data  Is  obtained  from  various  tests 
(tension,  compression,  and  shear)  and  conditionings  (hot,  cold,  hot-wet)  In 
order  to  obtain  the  value  ^ * . 


The  Pooling  Process 

In  order  to  effectively  apply  the  previously  discussed  methods,  a  moderately 
large  data  set  (n  >  30)  Is  suggested.  This  Is  accomplished  by  pooling  a  smaller 
set  S  (limited  available  data)  of  size  n$  with  a  larger  set  L  (from  prior  test¬ 
ing)  of  size  nL.  In  the  HK  process  the  objective  Is  to  represent  S  with  com¬ 
bined  data  sets  S  and  L  of  sample  Sp  with  size  m  ■  n^  +  n$  obtained  from  S  and 
L.  In  the  Ferguson's  Bayes  method  the  prior  Is  represented  by  L  and  the  empiri¬ 
cal  data  by  S.  The  Welbull  analysis  method  obtains  Information  for  estimating 
the  population  shape  parameter  by  pooling  S  and  L  and  using  S  to  determine 
the  scale  parameter 

Ideally,  If  the  mean  of  S  and  L  are  equal  and  their  variances  are  also 
equal,  then  pooling  could  easily  be  justified.  If  the  mean  values  differ,  then 
a  normalizing  procedure  combined  with  an  equality  of  variance  test  Is  required. 


1()6 


This  Involves  representing  S  and  L  by  the  distributions  S*  and  L*  with  means 


equal  to  zero,  that  Is,  let 


and 


S*  ■ 


L*  ■ 


s-s 

s 

L  ■[ 

I 


(16) 


where  S  and  L  are  means  of  the  S  and  L  data  values.  A  schematic  of  this 
transformation  Is  shown  In  Appendix  A.  The  variance  equality  between  S  and  L 
requirement  Is  stated  as, 

VAR[S*]  -  VAR[L*].  (17) 

In  order  to  determine  If  equality  of  variance  exists,  the  Siegel -Tukey 
nonparametrlc  rank  sum  method^13^  was  applied.  The  process  Involved  combining 
S*  and  L*  values  Into  a  single  ordered  array  and  assigning  low  ranks  to  extreme 

observations  and  high  ranks  to  central  observations.  That  Is,  assign  rank  1  to 

lowest  number  of  the  sequence,  ranks  2  and  3  to  the  two  highest  values,  then 

ranks  4  and  5  to  the  next  two  lowest.  This  process  Is  continued  until  all 

numbers  In  the  sequence  have  been  assigned  a  rank  Rj.  Note,  for  odd  number  of 
values,  the  middle  value  Is  not  counted  so  that  the  highest  rank  Is  even.  In 
the  ranking  process,  knowledge  of  the  assigned  values  from  $*  and  l*  must  be 
retained. 

The  sum  (Rj)  of  the  ranks  Is  obtained  for  either  S*  or  l*.  Usually,  the 
smaller  data  set  is  selected  In  order  to  reduce  the  amount  of  computation.  R$ 
obtained  from  the  S*  ordering  can  be  written  as, 


RS 


,  where  1  depends  on  ordering  of  ranking  for  S* 
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The  hypothesis  test  for  VAR[S*]  ■  VAR[L*]  Involves  applying  the  Wilcox  Rank  Sum 
(two  sample)  test^13-^  as  follows.  In  order  for  the  hypothesis  to  be  accepted, 
then 

R$  >  Rc,  where  Rc  Is  (18) 

a  critical  value  obtained  from  tables  In  Reference  (13).  The  values  are  tabu¬ 
lated  as  a  function  of  ns,  nL,  and  a  specified  significance  level.  For  example, 
a  10%  significance  level  would  represent  a  two  sided  test  with  a  5%  to  95% 
Interval  of  acceptance.  If  Rc  >  R$,  then  reject  the  hypothesis. 

An  example  showing  application  of  the  above  procedures  Is  as  follows. 

If  sample  from  S*:  5,  8,  10,  19,  25 
and  If  sample  from  L*:  4,  9,  12,  14,  16 

then 

Score  4  5  8  9  10  12  14  16  19  25 

Sample  L*  S*  S*  l*  S*  L*  L*  L*  S*  S* 

Rank  1  4  5  8  9  10  7  6  3  2 

and  Rs  ■  4  +  5  +  9  +  3  +  2  ■  23 

The  critical  value  from  the  tables  Is  Rc  «  19  for  ns  ■  nL  ■  5  with  significance 
level  of  10%  (a  two  sided  5%  level);  therefore,  since  Rs  >  Rc,  there  Is  no 
significant  difference  In  the  variances  between  S*  and  L*  at  the  10%  level. 

1 B 1  Basis  Values  for  S»  and  S  from  Pooled  Data 

Bayes  Solution 

If  the  transformation  (Equation  (16))  has  been  applied  and  the  equality  of 
variance  Is  established  between  S*  and  L*,  then  ' B '  values  can  be  obtained  using 
the  combined  data  sets  from  both  S*  and  L*.  In  the  Bayes  application  let  the 
smaller  sample  x  (newly  obtained  data)  to  be  represented  by  the  S*  values  and 
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the  larger  sample  t  (the  prior)  by  L*.  The  approximate  solution  to  i*  'vF(Sj) 
for  j  determines  the  jth  ordered  value  representing  the  ' B '  basis  value  (S*) 
when  u  Is  obtained  from  the  solution  in  Equation  (10). 

Nonparametrlc  HK  Method 

The  nonparametrlc^  solution  for  obtaining  'B*  basis  values  Involves  pool¬ 
ing  the  values  from  S*  and  L*  and  letting  the  resultant  array  of  values  be  x  In 

44- 

Equation  (14)  with  sample  size  m  ■  n$  +  nL.  Let  this  value  be  denoted  SB.  This 
method  Is  very  simple  to  apply  yet  provides  accurate  results  for  any  sample  size 
greater  than  2.  The  basis  values  for  S*  Is  not  sufficient  since  S  and  L  were 
the  original  data  sets  involved  in  the  analysis.  Therefore,  the  following 
transformation  Is  required! 

SB  “  SB  $.95  +  ^.95 

where  $B  Is  the  required  basis  value  for  the  small  sample  S.  The  S^g  values 
represent  the  lower  95%  confidence  value  for  the  mean  of  the  S  values.  The 
purpose  of  applying  S  55  was  to  prevent  a  situation  where  the  required  95% 
coverage  was  not  obtainable  due  to  variability  In  estimating  the  mean  $  of  S. 

values  were  consistently  less  than  the  10%  point,  95%  of  the  time  (defi¬ 
nition  of  ' B '  values).  The  results  from  both  simulated  and  actual  test  data 
showed  on  occasion  excessively  low  SB  values,  but  the  coverage  was  maintained 
which  was  not  necessarily  so  when  S  was  substituted  for  T^g  In  Equation  (19). 
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Wei  bull  Method  -  Estimating  Population  Shape  Parameter 
In  Equation  (15)  the  pooled  estimate  &(*  Is  obtained  from  the  maximum 
likelihood  estimator  (MLE)  using  combined  data  sets  and  ,  where 

S^j  ■  S/|3S  and 

*  »  V*  ’ 

/3j  and  /3L  are  the  Welbull  scale  parameter  estimates  for  S  and  L  respectively* 
If  a  basis  value  for  S  Is  required,  then,  after  the  Initial  equality  of  variance 
between  and  Is  established,  the  value  can  be  obtained  from  the  follow¬ 
ing: 

•B's  -  /JjC-  ln(0.90)]1/  £**  [2n /  ^ ,95(2n)]1/  <**  (20) 

In  Appendix  B  a  flowchart  Is  shown  summarizing  the  ordered  procedures  required 
for  applying  methods  described  In  this  report. 

Results  and  Discussions 

In  Figure  3A,  the  ' B *  basis  value  results  are  shown  for  the  small  sample  S 
with  or  without  a  contribution  In  the  variance  estimate  from  large  sample  L.  NB 
is  the  nonparametrlc  Bayes  solution  where  the  prior  Is  represented  by  L  with 
sample  size  of  nL  ■  30  and  empirical  data  by  S  with  size  ns  ■  6.  The  HK  values 
represent  the  modified  Hanson-Koopmans  method  results  where  L  and  S  are  pooled. 
W(6)  and  W( 36 )  represents  the  property  values  obtained  from  the  Welbull  analy¬ 
sis^-2*  with  a  sample  of  6  and  36  respectively  from  S.  The  HK(36)  was  deter¬ 
mined  from  the  modified  Hanson-Koopmans  method  with  36  values  from  S  only.  The 
data  was  obtained  from  randomly  selecting  samples  from  Welbull  distributions 
with  scale  ■  100,  and  a  randomly  selected  shape  parameter  from  a  uniform 
distribution.  The  range  of  values  were  from  5  to  25  for  L.  The  S  distri¬ 
bution  shape  parameter  was  15.  By  Introducing  the  variability  In  -<L, 
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evaluation  of  the  equality  of  variance  test  can  be  made  with  respect  to  accepta¬ 
bility  in  pooling  of  L  and  S  and  using  L  as  a  prior  In  the  methods.  The  NB,  HK, 
and  W(36)  ' B*  basis  values  show  the  required  coverage,  that  Is,  95%  of  the  'B' 
values  are  equal  to  or  less  than  the  10%  point  of  the  population  distribution. 

The  results  are  exceptionally  good  In  that  they  are  almost  exactly  what  Is 
required  without  being  either  overly  conservative  or  nonconservative  In  addition 
to  verifying  the  acceptability  of  the  equality  variance  test.  W(6)  and  HK(36) 
are  slightly  conservative  because  of  the  lower  estimate  measured  from  the  true 
10%  point.  The  W(6)  ■  82.2  Is  not  excessively  conservative  for  n$  ■  6,  although 
using  data  from  a  Welbull  model  could  have  been  helpful.  The  Wei  bull  result 
W( 36 )  should  be  a  good  estimate  since  the  36  values  came  from  a  known  Welbull 
model  ex  ■  15,  •  100.  The  fact  that  NB  and  HK  provided  equally  good  results 

shows  the  methods  can  provide  'B'  values  for  a  small  sample,  which  are  equiva¬ 
lent  to  those  values  obtained  from  a  much  larger  Welbull  sample,  when  pooling 
the  large  and  small  samples.  N(6),  basis  value  obtained  from  the  normal 
model^-1,  2‘  3*  result  Is  nonconservative  with  ' B 1  »  87.5,  although  It  is  not 
excessive  for  n  «  6. 

In  Figure  3B  the  range  of  ' B '  value  results  are  shown  with  HK( 36)  and  W(36) 
having  significantly  less  variability  than  the  other  methods.  This  Is  due  to 
the  much  larger  sample  size  which  Is  available  to  estimate  directly  the  ' B 1 
value.  W(6)  and  N(6)  have  an  excessively  large  amount  of  variability  (a  range 
of  45  to  95)  which  Is  due  to  the  Instability  (see  Figures  1A  and  IB)  In  estimat¬ 
ing  the  parameters  required  to  compute  basis  values  from  the  small  sample  sizes. 
The  NB  and  HK  solution  show  less  variability  (a  range  of  55  to  92)  than  W(6)  and 
N(6),  but  Is  still  quite  large  due  to  the  Inability  to  estimate  the  mean  value 
from  S  for  6  data  values  when  the  reverse  transformation  Is  made  In  Equation 
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(19).  If  the  normalization  process  was  not  required  (L  and  S  from  the  same 
population),  then  variability  In  NB  and  HK  can  be  reduced  by  applying  pooled 
non-normal  1  zed  data  to  obtain  mean  value  In  the  reverse  transformation. 

The  experimental  test  results  used  In  the  analysis  showhthe  following  fig¬ 
ures  were  obtained  from  published  data  In  [14],  In  Figure  4A,  both  S  and  L  are 
from  the  same  population  of  short  beam  shear  tests  applied  to  Narmco  composite 
material.  The  NB  and  HK  basis  values  were  obtained  by  substituting  the  5  g5 
value  In  Equation  (19)  by  JUn  from  Equation  (6)  for  the  reverse  transformation 
described  In  Equation  (16).  The  results  in  Figure  4A  show  excellent  agreement 
between  the  10*  point  and  the  95th  percentile  of  the  basis  values  for  NB,  HK, 
N(6) ,  and  HK(36). 

In  Figure  4B,  variability  In  the  basis  value  for  NB  and  HK  Is  very  small 
relative  to  W(6)  and  N(6)  Indicating  the  desirability  of  the  proposed  nonpara- 
metric  methods.  With  S  and  L  from  the  same  population  permlttiwj  the  appllca- 

A 

tlon  of /U  ^  ^  Equation  (19).  This  resulted  In  a  more  stable  less  conservative 
basis  value  since  the  Bayes  est1matory£^n  Is  determined  from  more  Information 
than  S. 

In  Figure  5A  the  ' B *  basis  values  were  obtained  from  1,000  random  samples  of 
short  beam  shear  test  results.  The  material  was  manufactured  by  the  Hercules 
Co.,  Utah.  In  all  cases  the  coverage  was  obtained  (all  ' B *  values  less  than  the 
10%  point),  although  they  were  somewhat  conservative.  NB  and  HK  were  approxi¬ 
mately  9.2  where  the  10*  point  Is  9.8.  The  W(6)  value  of  8.6  was  exceptionally 
low.  The  6%  difference  between  10*  point  and  the  basis  value  obtained  from  NB 
and  HK  analysis  Is  much  better  than  the  14*  difference  determined  from  W(6). 

In  Figure  5B  the  ranges  are  much  smaller  for  NB,  HK,  and  HK(36)  when  com¬ 
pared  to  W(6)  and  N(6),  which  again  shows  the  advantage  of  having  more  data 
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available  by  combining  the  L  and  S  samples.  A  low  value  of  7  and  the  highest  of 
9.5  from  NB  and  HK  Is  much  better  than  4.2  to  9.2  from  W(6)  method. 

The  NB  and  HK  methods  were  very  effective  as  shown  In  Figure  $A  In  obtaining 
a  'B'  basis  value  for  crossply  tension  test  results  (manufactured  by  Narmco) 
when  combining  with  prior  available  unidirectional  tension  data.  The  N(6) 
estimate  also  provided  desirable  coverage.  Figure  6B  shows  less  variability  In 
basis  value  estimates  for  the  NB  and  HK  procedures  when  compared  to  the  small 
sample  W(6)  and  N(6)  methods.  The  reduction  In  the  variability  In  the  'B1  value 
Is  the  primary  advantage  of  using  NB  and  HK  methods  when  compared  to  the  W(6) 
and  N(6)  analysis  results,  although  reducing  the  model  Identification  Issue  Is 
also  desirable. 

Figure  7A  shows  excellent  results  for  the  NB  and  HK  by  providing  'B'  values 
almost  equal  to  the  required  10%  point.  W(6)  basis  value  Is  overly  conserva¬ 
tive,  and  N(6)  and  HK( 36)  do  not  meet  the  required  coverage  rate  by  having 
values  somewhat  greater  than  the  10%  point.  The  results  demonstrate  the  advan¬ 
tage  of  the  NB  and  HK  methods  being  a  nonparametrlc  procedure  In  determining 
basis  values  for  data  obtained  from  relatively  expensive  (unidirection  small 
sample)  test  methods  by  pooling  the  data  with  less  expensive  (short  beam  shear) 
coupon  test  results. 

Figure  7B  shows  the  relatively  small  variability  In  the  * B 1  estimate  from  NB 
and  HK  and  HK(36)  when  compared  to  the  small  sample  procedures  W(6)  and  N(6). 

The  * B '  basis  results  from  the  crossply  compression  data  using  additional 
data  obtained  from  a  very  simple  and  Inexpensive  short  beam  shear  testing  proce¬ 
dures  are  shown  In  Figure  8A.  The  NB  and  HK  methods  show  less  conservative  1 B ' 

» 

estimates  than  W(6).  The  ' B *  values  from  W(6)  are  somewhat  conservative  with 
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N(6)  results  showing  good  agreement  with  the  10%  point.  The  variability  In  '8' 
values  In  Figure  8B  Is  similar  to  prior  results  In  Figure  7B. 

The  crossply  compression  * B 1  values  from  sample  of  6  Narmco  material  using 

the  additional  30  short  beam  shear  data  values  from  the  material  made  by  the 

Hercules  Co.  are  shown  In  Figure  9A.  The  NB  and  HK  results  are  about  5%  below 

the  10%  point  which  is  not  excessively  low.  Although  excellent  coverage  was 

obtained  for  N(6),  extensive  variability  In  the  basis  value  results  Is  shown  In 
Figure  9B. 

The  results  In  Figure  10A  show,  for  large  sample  of  90  and  small  sample  of  6 
obtained  from  random  Wei  bull  samples,  excellent  agreement  between  the  10%  point 
and  the  estimated  ' B '  basis  values.  The  exception  Is  the  SHB  method  (see  Equa¬ 
tion  (20))  which  falls  to  provide  the  proper  coverage.  This  result  Is  probably 

* 

due  to  the  Instability  In  estimating  scale  parameter /*  from  a  small  sample  of  6 
values.  If  a  lower  95%  confidence  was  obtained  for  ft  ,  then  the  SHB  method 
could  possibly  provide  satisfactory  results.  The  similar  results  shown  for  both 
W(6)  and  W(96)  show  that  If  the  Welbull  analysis,  even  for  small  samples,  Is 
applied  to  data  from  the  Welbull  population  of  data,  accurate  • B*  basis  values 
can  be  obtained  that  provide  the  coverage  and  also  are  not  overly  conservative. 
Results  In  Figure  10B  show  a  value  of  42  for  W(6)  could  be  obtained  compared  to 
the  85  for  the  95  percentile.  Although  acceptable  coverage  for  W(6)  was 
obtained  which  was  possibly  due  to  the  use  of  known  Welbull  data,  the  variabil¬ 
ity  Is  substantial  In  the  basis  value  estimate.  Since  It  Is  Impossible  to 
correctly  Identify  models  for  small  samples,  It  Is  suggested  the  nonparametrlc 
methods  (NB  and  HK)  In  this  paper  be  applied  since  the  methods  do  not  require 
prior  knowledge  of  the  assumed  distribution.  The  SHB  method  shows  relatively 
low  variability  In  estimating  the  basis  value  but  does  require  knowledge  of 
distribution  (eg.,  Welbull jnormal ). 
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Conclusions 

Efficient  tolerance  limits  ( 'B'  values)  can  be  obtained  for  small  samples  of 
composite  material  strength  data  when  pooled  with  a  much  larger  data  set  of 
similar  material  but  obtained  from  potentially  different  mechanical  properties. 
Ideally,  both  small  and  large  data  sets  should  be  obtained  from  the  same  test 
and  material  In  order  to  maximize  the  effectiveness  of  the  proposed  methods. 

The  proposed  methodology  resulted  In  reducing  the  overconservatism  and  excessive 
variability  obtained  from  using  only  the  smaller  sample  In  computing  the  toler¬ 
ance  limit. 
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Table  I:  M  and  u  Values  for  Bayesian  Basis  Value  Computation 


M 

U 

M 

u 

M 

U 

M 

U 

1 

0.021953 

51 

0.044804 

101 

0.057686 

151 

0.064302 

2 

0.017855 

52 

0.045192 

102 

0.057856 

152 

0.064395 

3 

0.016529 

53 

0.045565 

103 

0.058023 

153 

0.064514 

4 

0.016140 

54 

0.045937 

104 

0.058188 

154 

0.064609 

5 

0.016199 

55 

0.046301 

105 

0.058352 

155 

0.064717 

6 

0.016516 

56 

0.046648 

106 

0.058517 

156 

0.064814 

7 

0.016997 

57 

0.046996 

107 

0.058670 

157 

0.064912 

6 

0.017590 

58 

0.047339 

108 

0.058837 

158 

0.065010 

9 

0.018264 

59 

0, 04^673 

109 

0.059006 

159 

0.065099 

10 

0.018996 

60 

0.048011 

110 

0.059156 

160 

0.065193 

11 

0.019769 

61 

0.048318 

111 

0.059313 

161 

0.065273 

12 

0.020570 

62 

0.048642 

112 

0.059454 

162 

0.065382 

13 

0.021391 

63 

0.048945 

113 

0.059619 

163 

0.065462 

14 

0.022223 

64 

0.049255 

114 

0.059761 

164 

0.065555 

15 

0.023060 

65 

0.049563 

115 

0.059914 

165 

0.065658 

16 

0.023897 

66 

0.049848 

116 

0.060051 

166 

0.065734 

17 

0.024729 

67 

0.050144 

117 

0.060192 

167 

0.065822 

lb 

0.025554 

68 

0.050421 

118 

0.060344 

168 

0.065910 

19 

0.026368 

69 

0.050695 

119 

0.060480 

169 

0.065996 

20 

0.027171 

70 

0.050968 

120 

0.060628 

170 

0.066108 

21 

0.027959 

71 

0.051238 

121 

0.060754 

171 

0.066192 

22 

0.028734 

72 

0.051506 

122 

0.060883 

172 

0.066277 

23 

0.029491 

73 

0.051771 

123 

0.061031 

173 

0.066384 

24 

0.030233 

74 

0.052034 

124 

0.061162 

174 

0.066449 

25 

0.030959 

75 

0.052284 

125 

0.061292 

175 

0.066530 

26 

0.031666 

76 

0.052530 

126 

0.061420 

176 

0.066613 

27 

0.032361 

77 

0.052773 

127 

0.061547 

177 

0.066705 

28 

0.033033 

78 

0.053017 

128 

0.061679 

178 

0.066789 

29 

0.033695 

79 

0.053244 

129 

0.061802 

179 

0.066872 

30 

0.034339 

80 

0.053479 

130 

0.061933 

180 

0.066934 

31 

0.034967 

81 

0.053702 

131 

0.062065 

181 

0.067007 

32 

0.035577 

82 

0.053932 

132 

0.062179 

182 

0.067098 

33 

0.036172 

83 

0.054160 

133 

0.062293 

183 

0.067176 

34 

0.036754 

84 

0.054375 

134 

0.062430 

184 

0.067258 

35 

0.037328 

85 

0.054600 

135 

0.062553 

185 

0.067333 

36 

0.037884 

86 

0.054808 

136 

0.062667 

186 

0.067418 

37 

0.038420 

87 

0.055017 

137 

0.062784 

187 

0.067486 

38 

0.038952 

88 

0.055221 

138 

0.062894 

188 

0.067569 

39 

0.039461 

89 

0.055435 

139 

0.063010 

189 

0.067628 

40 

0.039964 

90 

0.055634 

140 

0.063128 

190 

0.067720 

41 

0.040459 

91 

0.055831 

141 

0.063245 

191 

0.067794 

42 

0.040944 

92 

0.056024 

142 

0.063344 

192 

0.067871 

43 

0.041409 

93 

0.056215 

143 

0.063459 

193 

0.067952 

44 

0.041864 

94 

0.056417 

144 

0.063550 

194 

0.068022 

45 

0.042314 

95 

0.056599 

145 

0.063666 

195 

0.068103 

46 

0.042751 

96 

0.056781 

146 

0.063763 

196 

0.068178 

47 

0.043182 

97 

0.056960 

147 

0.063899 

197 

0.068237 

48 

0.043596 

98 

0.057153 

148 

0.063985 

198 

0.068315 

49 

0.044009 

99 

0.057332 

149 

0.064101 

199 

0.068388 

50 

0.044413 

100 

0.057502 

150 

0.064197 

200 

0.068459 
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Table  II:  Modified  Hanson-Koopmans  Constants  for  Basis  Value 


rh 

r 

8 

Jc 

2 

1 

2 

35.177 

3 

1 

3 

7.859 

4 

1 

4 

4.505 

5 

1 

4 

4.101 

6 

1 

5 

3.064 

7 

1 

5 

2.858 

8 

1 

6 

2.382 

9 

1 

6 

2.253 

10 

1 

6 

2.137 

11 

1 

7 

1.897 

12 

1 

7 

1.814 

13 

1 

7 

1.738 

14 

1 

8 

1.599 

15 

1 

8 

1.540 

16 

1 

8 

1.485 

17 

1 

8 

1.434 

18 

1 

9 

1.354 

19 

1 

9 

1.311 

20 

1 

10 

1.253 

21 

1 

10 

1.218 

22 

1 

10 

1.184 

23 

1 

11 

1.143 

24 

1 

11 

1.114 

25 

1 

11 

1.087 

26 

1 

11 

1.060 

27 

1 

11 

1.035 

28 

1 

12 

1.010 

29 

1 

— 

1 

30 

2 

12 

1.373 

31 

2 

12 

1.344 

32 

2 

12 

1.315 

33 

2 

13 

1.270 

34 

2 

13 

1.245 

35 

2 

13 

1.221 

36 

2 

13 

1.197 

37 

2 

13 

1.174 

38 

2 

13 

1.151 

39 

2 

13 

1.129 

40 

2 

13 

1.108 

41 

2 

14 

1.083 

42 

2 

14 

1.064 

43 

2 

14 

1.045 

44 

2 

14 

1.027 

45 

2 

14 

1.009 

46 

2 

-- 

1 
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Figure  4 B  :  The  Range  of  'B'  Values 


Monte  Carlo  Study  -  Hercules  Short  Beam  Shear  Data 

L  and  S  From  Same  Population 


Figure  5A  :  Coverage  Evaluation  Figure  5B  :  The  Range  of  "B'  Values 


j  Carlo  Study  -  Narmco  Data 


Figure  7A  :  Coverage  Evaluation  Figure  7B-:  The  Range  of  *8’  Values 
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L :  Short  Beam  Shear  Results = 30 
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Weibull  Data 


Figure  10A:  Coverage  Evaluation  Figure  1 0B:  The  Range  of ’B’ Values 


TARGET  PRIORITIZATION  TO  OPTIMIZE  EXPECTED 
UTILITY  BASED  ON  RANDOM  FIRE 


Ann  EM.  Brodeen 
Director 

U.S.  Army  Ballistic  Research  Laboratory 


ATTN:  SLCBR-SE-W 


Aberdeen  Proving  Ground,  MD  21005-5066 
(301)  278-8947,  AV 298-8947 


Douglas  H.  Frank 

Indiana  University  of  Pennsylvania 
208  Stright  Hall 
Dept,  of  Mathematics 
Indiana,  PA  15705-1072 
(412)357-3793 


Abstract 


The  U.S.  Army  Ballistic  Research  Laboratory  has  been  research¬ 
ing  innovative  methodologies  directly  applicable  to  the  problem  of  tar¬ 
get  value  analysis  (TVA),  i.e.,  assigning  values  to  targets  for  the  purpose 
of  developing  an  optimal  target  engagement  ordering.  The  problems 
associated  with  assigning  target  values  as  an  aid  in  target  selection  have 
been  examined  by  many  approaches,  by  both  other  government  agen¬ 
cies  and  contractors.  Here  the  question  is  examined  from  the  standpoint 
of  optimizing  some  utility  function.  Expected  values  of  utility  functions 
are  derived  in  terms  of  the  two  chosen  research  parameters,  rate  of  fire 
and  probability  of  kill,  using  stochastic  techniques.  In  the  case  of  two 
enemy  targets,  an  optimal  ordering  for  all  utilities  investigated  is 
obtained.  Some  special  cases  of  utilities  are  considered  and  the  results 
generalized  for  an  arbitrary  mix  of  targets. 
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1.  Introduction 


The  problem  of  assigning  values  to  targets  as  an  aid  in  target  selection  has  been  exam¬ 
ined  for  over  a  decade  by  many  approaches,  e.g.,  Fire  Support  Mission  Area  Analysis,  US 
Army  Material  Systems  Analysis  Activity  military  worth  study,  classification  tree  methodol¬ 
ogy  (another  approach  being  pursued  by  the  US  Army  Ballistic  Research  Laboratory  (BRL)). 
During  the  summer  of  1988,  Dr.  Douglas  H.  Frank,  Associate  Professor,  Department  of 
Mathematics,  Indiana  University  of  Pennsylvania,  worked  with  Ann  E.M.  Brodeen,  one  of  the 
BRL’s  principal  target  value  analysis  (TVA)  Investigators,  to  consider  a  probabilistic 
approach  to  the  problem.  (Dr.  Frank  was  assigned  to  the  BRL  under  the  sponsorship  of  the 
US  Army  Summer  Faculty  Research  and  Engineering  Program.) 

Target  values  are  assessments  keyed  to  the  enemy's  perception  of  the  functions  of  its 
assets;  TVA  is  the  methodology  which  identifies  potential  high  value  target  sets,  /.&,  assets 
which  the  enemy  threat  commander  requires  for  the  successful  completion  of  his  mission, 
within  the  given  tactical  scenario.  These  targets,  if  successfully  countered,  can  provide  the 
friendly  force  with  a  tactical  opportunity.  Although  the  TVA  process  may  include  complex 
algorithms,  it  should  be  simple  enough  for  the  user,  Le.,  the  soldier,  to  understand.  Simply 
put,  he  must  be  able  to  influence  the  process  in  order  to  meet  the  specific  needs  of  his  com¬ 
mander.  For  the  field  artillery  to  remain  responsive,  the  soldier  must  be  able  to  change  target 
priorities  as  quickly  as  the  tactical  situation  changes  and  be  able  to  interpret  the  overall 
impact  that  such  changes  may  have  on  the  outcome  of  the  operation. 

Although  TVA  is  a  very  subjective  issue,  the  intent  of  this  research  was  to  show  that 
assigned  target  values  can  be  based  on  mathematical  models.  Two  objectives  were  defined  for 
the  proposed  study:  1)  define  a  value  for  each  enemy  target  in  a  target  array  such  that  a  tar¬ 
get  engagement  sequence  can  be  determined,  and  2)  evaluate  the  target  engagement 
sequence  from  the  standpoint  of  optimizing  an  expected  utility  function  based  on  a  desired 
tactical  outcome. 

Details  of  the  BRL’s  probabilistic  approach  to  target  value  analysis  based  on  a  random 
battle  scenario  are  outlined  in  this  paper. 

2.  The  Battle 

In  our  earlier  research,  we  considered  target  engagement  orderings  to  maximize  the  out¬ 
comes  of  a  simple  battle  in  which  both  the  friendly  fire  unit  and  the  enemy  targets  fired 
simultaneously.  Here,  battle  outcome  probabilities  and  optimal  engagement  orderings  are 
considered  in  a  manner  similar  to  that  of  the  simple  battle  [1,2], 

The  random  battle  is  between  a  single  friendly  fire  unit  and  a  group  of  enemy  targets. 
Each  enemy  target  as  well  as  the  friendly  fire  unit  fire  independently  and  at  a  rate  of  fire  that 
is  exponentially  distributed  with  different  mean  rates  and  removal,  i.e.,  kill,  probabilities.  The 
kill  probabilities  are  constant  from  volley  to  volley  for  both  the  fire  unit  and  each  enemy  tar¬ 
get. 
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2.1  Parameter  Selection 


Subject  matter  discussions  held  with  MAJ  William  T.  Dougherty,  Field  Artillery  Coordi¬ 
nation  Officer  assigned  to  the  BRL  when  the  TVA  probabilistic  approach  was  initiated,  led  to 
the  selection  of  target  vulnerability  and  target  threat  as  the  parameters  of  interest.  When 
considering  an  enemy  target’s  value,  it  is  natural  to  characterize  this  value  by  the  ability  of  the 
friendly  fire  unit  to  remove  the  enemy  target  within  some  time  frame,  £«.,  target  vulnerabil¬ 
ity,  as  well  as  by  the  ability  of  the  enemy  target  to  achieve  its  objective  within  that  same  time 
frame,  ie.,  threat.  Removal  of  the  enemy  target  Is  considered  to  be  either  its  complete  des¬ 
truction  or  the  infliction  of  a  level  of  damage  severe  enough  to  abate  the  target’s  contribution 
to  the  enemy  force  given  some  particular  tactical  scenario.  The  objective  of  the  enemy  target 
might  also  be  either  the  destruction  of  the  friendly  fire  unit  or  the  infliction  of  a  severe  level 
of  damage  upon  it.  (It  should  be  noted  that  the  definitions  of  the  parameters  developed  by 
the  principal  investigators  are  in  the  interest  of  the  research  and  may  not  be  in  accordance 
with  those  of  the  field  artillery  community.) 

2.2  Two  Target  Battle 


We  first  derive  results  for  a  battle  with  two  enemy  targets  and  then  extend  the  results  to 
T  targets. 

Consider  the  following  parameters  for  targets  /  *  1, 2 : 

PB(  -  probability  of  friendly  fire  unit  removing  target  / 

P^  ■  probability  of  friendly  fire  unit  being  removed  by  target  i 

■  mean  rate  of  fire  of  friendly  fire  unit  against  target  i 

pt  ■  mean  rate  of  fire  of  target  /  against  friendly  fire  unit 

®/  -  /J(PB  ■  friendly  fire  unit  firepower  against  target/ 

( vulnerability  of  target  / ) 

R/  “  P/Pr,  ■  target  /  firepower  against  friendly  fire  unit 
( threat  of  target  / ) 


We  first  show  that  the  results  of  the  classic  stochastic  duel  can  be  extended  to  a  battle 
between  a  single  friendly  fire  unit  and  two  enemy  targets  [3,4],  We  assume  target  1  is  engaged 
until  it  is  removed  before  target  2  is  engaged.  The  battle  concludes  when  either  the  friendly 
fire  unit  is  removed  or  it  has  removed  both  targets.  For  our  purposes  we  define  a  victory  as 
the  removal  of  both  targets  regardless  of  whether  or  not  the  fire  unit  survives. 
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If  a  target  w  has  a  mean  rate  of  fire  r  and  a  kill  probability  p,  then  the  probability  of  not 
killing  the  target  in  some  time  h  is 


(1) 


Eq.  (1)  will  hereafter  be  referred  to  as  Lemma  1.  The  following  proof  is  offered. 

Let  X  be  the  number  of  volleys  fired  during  some  time  h.  Then  X  has  a  Poisson  distribu¬ 
tion  with  mean  rh. 


Ow(h)  -  £ 

*■0 


erH(rh)x 

x! 


(1  -P)* 


me-rherH(i.p) 
-  erhp . 


(2) 


Our  Lemma  2  states  that  in  a  battle  with  two  enemy  targets,  the  probability  of  the  friendly 
fire  unit  removing  target  1  before  being  removed  is 


P[1 1  NOT]  ■  -  ‘  ~  ■  (3) 

Bj  +  Rj  +  Rj 


Lemma  2  is  proved  as  follows.  Divide  the  time  of  battle  into  units  of  length  h  and  consider  the 
absorbing  Markov  process  formed  [5].  From  this  process  the  probability  of  the  event  occur¬ 
ring  is 


00 


P*[1 1  NOT]  -  ([1-Qb,(/.)]<V'0Qr1W)  s  (Qs.WQr.WQr.W) 


i-O 


WQb.C'OQr^OQr,^))  ' 


(4) 
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Applying  Lemma  1,  we  obtain 


P4[1|H0T] 


6 _ ~€ _ 


(5) 


Vl 


Bt 


B, 


P[1 1  NOT]  -  limPJl  |  NOT]  -  — 

*V*V*Pii.  B^Rj  +  R* 


(6) 


In  a  battle  between  the  friendly  fire  unit  and  target  2,  the  probability  of  destroying  the 
target  is 


P[2] 


B» 

B2<V 


(7) 


Lemma  3  is  a  well-known  result  [4]. 

Theorem  1  defines  w,  the  number  of  targets  destroyed.  The  density  of  w  is 


f(0) 


V*2 

B1  +  Bj  +  ^2 


f(l) 


B1R2 

(B^Rj  +  RjXBj  +  Rj)  ’ 


f(2) 


BiB2 

(B1  +  R1  +  R2)(B2  +  R2) 


(8) 


The  event  w  -  0  is  the  complement  of  the  event  stated  as  Lemma  1.  Because  the  battle  is  a 
Markov  process  we  have 


f(l)  -P[l|NOT]-(l-P[2])  and  f(2)  -  P[1 1  NOT]-P[2] .  (9) 


The  result  follows  from  Lemmas  2  and  3. 
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2,3  Multi  Target  Battle 


Suppose  there  are  T  targets  which  are  engaged  in  numerical  order.  We  extend  the 
definitions  of  B,  and  R,  for  i  ■  1, 2, ....  T. 

T 

Define  S„-0,  and  B0  *  1.  (10) 


The  proof  of  Theorem  1  can  be  easily  generalized.  If  w  is  the  number  of  targets  removed  in  a 
battle  with  T  targets,  Theorem  2  states  that  the  density  of  w  is 


f(H0« 


(  w 

n 

N-0 

T 

it 

/-i 


— ) 
B/  +  s/« 

B  / 

B  i  +  S, 


: 


B>V  +  1  +  1 

if  w-T 


if  w<T 


(11) 


3.  Evaluation  Criteria 

In  general,  decision  makers  such  as  gamblers,  baseball  managers,  insurance  companies,  and 
others  engage  in  what  is  colloquially  refered  to  as  "playing  the  percentages",  characterized  by 
a  preference  for  the  optimal  act  that  yields  the  greatest  long-run  average  profit.  That  is,  the 
optimal  act  is  the  one  that  would  result  in  the  largest  long-run  average  profit  if  the  same  deci¬ 
sion  were  to  be  made  repeatedly  under  the  same  conditions;  as  the  number  of  repetitions 
becomes  large,  the  observed  average  payoff  approaches  the  theoretical  expected  payoff.  How¬ 
ever,  many  important  decisions  are  made  under  unique  sets  of  conditions,  and  it  may  not  be 
realistic  to  think  in  terms  of  many  repetitions  of  the  same  decision  situation.  Indeed,  many  of 
the  field  artillery  commander’s  most  important  decisons  are  unique,  high-risk  situations, 
whereas  less  important,  routine  decisions  are  ones  that  may  be  delegated  to  subordinates. 
Therefore,  it  is  useful  to  have  an  apparatus  for  dealing  with  one-time  decision  making. 

Utility  theory  provides  such  an  apparatus,  as  well  as  providing  a  logical  method  for 
repetitive  decision  making.  The  term  "utility"  as  conceived  by  Von  Neumann  and  Morgen- 
stern  [1947]  is  a  measure  of  value  used  in  the  assessment  of  situations  involving  risk,  which 
provides  a  basis  for  decision  making.  Different  sets  of  axioms  that  imply  the  existence  of  utili¬ 
ties  with  the  property  that  expected  utility  is  an  appropriate  guide  for  consistent  decision 
making  are  presented  in  Von  Neumann  and  Morgenstem  [1947],  Savage  [1954],  Luce  and 
Raiffa  [1957],  Pratt,  Raiffa,  and  Schlaifer  [1965],  and  Fishbum  [1970], 
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3.1  Construction  of  Utility  Functions 


The  different  algorithms  for  determining  target  values  do  not  always  yield  the  same  tar* 
get  engagement  ordering.  This  poses  the  obvious  question  of  which  approach  should  be  used. 
The  desired  approach  would  be  the  one  whose  target  ordering  provides  the  "best"  result. 
However,  if  "best"  is  interpreted  as  "total  victory",  and  if  T,  the  total  number  of  enemy  targets, 
is  large,  then  "victory"  for  a  single  friendly  fire  unit  would  quite  likely  be  a  rare  event.  Thus, 
additional  criteria  shall  be  considered  for  assessing  "best"  results. 

Recall  that  the  overall  objective  is  to  assign  a  value  to  each  enemy  target  to  determine 
the  order  in  which  to  engage  the  targets.  This  order  should  be  chosen  to  maximize  some 
desired  result  of  the  battle.  Therefore,  consider  a  utility  function,  U,  of  the  number  of  targets 
removed,  W,  by  the  friendly  fire  unit  during  the  battle.  This  function  should  depend  on  the 
battlefield  scenario  as  well  as  the  desired  battle  objective  of  the  friendly  fire  unit.  Assume 
that  U(W)  will  be  non-decreasing,  U(0)  ■  0  and  U(T)  ■  1. 

Generally,  U(H)  is  assigned  over  a  continuous  range  of  possibilities;  however,  special 
liberty  has  been  taken  in  the  analysis  of  the  utility  functions  discussed  below.  Since  each  of 
these  utility  functions  is  based  on  the  mathematical  model’s  assumption  that  an  enemy  target 
either  survives  or  is  completely  removed  from  the  battle,  these  functions  are  evaluated  only  at 
discrete  points. 

3.2  Utility  Based  on  Total  Victory 


If  the  goal  is  to  remove  all  enemy  targets,  then  we  wish  to  maximize  f(T).  This  is  an 
extreme  example  of  a  convex  funtion. 


U1(W) 


if  W <T 
if  W-T. 


(12) 


Note  that  f(T) 


(Bj  +  S^-tBp  +  Sr) 


Since  the  numerator  is  the  same  for  all  permutations,  we  must  minimize  the  quantity, 
T 

n<B,+s,). 

i-i 
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3.3  Utility  Based  on  the  Number  of  Hits 


If  hitting  all  targets  is  not  essential,  and  all  targets  are  equally  important,  we  may  wish  to 
maximize  the  number  of  targets  removed. 


U2(W) 


W 

T 


(13) 


T  W 

For  utility  two  we  wish  to  maximize  £  — ,  for  f(w) . 

T 

H*  ■  1 


3.4  Utility  Based  on  a  Reduction  in  Threat 

Lemma  2  shows  that  the  combined  threat  of  an  array  of  targets  acting  as  a  single  target 
is  the  sum  of  their  individual  threats.  Thus,  in  removing  w  targets,  we  reduce  the  overall 
threat  from  to  Sw  + 1 


U3(W)  -  1- 


(14) 


where  we  define  Sj.  +  j  ■  0  .  Note  that  U3(  W)  depends  on  prioritizing  the  targets  according 
to  VAL  3,  whereas  utilities  1,  2  and  4  are  not  dependent  on  any  specific  target  value  algo¬ 
rithm. 

3.5  Inflective  Shaped  Utility 


In  many  battles  the  enemy  can  be  halted  when  it  loses  only  a  small  proportion  of  its 
forces.  In  these  cases  an  inflective  utility  function  seems  appropriate.  For  convenience  we 
consider  an  extreme  example  of  an  inflective  type  function. 


U4(W) 


if  W<.3T 
if  W  >  .3T 


(15) 
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4.  Values  Based  on  Two  Targets 

For  two  enemy  targets  there  are  only  two  possible  permutations:  1,2  or  2,1.  The  utility  U 
will  be  determined  by  the  utility  for  target  1,  that  is,  17(1).  We  will  derive  a  generic  target 
value  algorithm  for  all  utility  functions  and  examine  the  four  special  cases  from  Section  3. 

4.1  General  Optimal  Results 

Suppose  U(  1)  ■  c  is  the  same  for  either  target  permutation.  Let  EJU]  and  E2[U]  be 
the  expected  utilities  for  the  orderings  1,2  and  2,1,  respectively.  From  Theorem  1 


Bt[U]  - 


B^cRj  +  Ba) 

(B1  +  R1+R2)(B2  +  R2) 


Ba[U] 


BaCcR^Bt) 

(B2  +  R1  +  R2)(B1  +  R1) 


Setting  E,  [U]  >  EJU]  and  simplifying,  we  obtain  an  inequality  concerning  the  threat  and 
vulnerability  for  each  target. 

We  now  state  Theorem  3.  E* [U]  >  E^U]  if  and  only  if 


B1(B1  +  R1) 


B2(B2  +  R2) 


(Bj  +  SHcRj  +  Bj)  (Ba  +  S)(cR2  +  B2) 


where  S  -  Rj  + is  a  constant  representing  total  enemy  firepower. 

Theorem  3  gives  us  a  generic  target  value  algorithm  for  utility  c.  The  value  of  a  target 
with  threat  R  and  vulnerability  B  relative  to  a  utility  c  is 


B(B  +  R) 

- i - 1 —  #  (18) 

(B  +  S)(cR  +  B) 

The  shortcomings  of  this  definition  are  the  presence  of  S  which  depends  on  the  entire 
array  of  targets  and  the  limitation  to  utilitites  which  are  independent  of  target  orderings.  One 
approach  is  replace  S  by  2R  (where  S  is  based  upon  knowledge  about  target  1  only).  We  do 
not  recommend  this  since  optimality  is  not  guaranteed.  We  will  see  in  some  of  the  special 
cases  that  an  equivalent  value  without  S  can  be  obtained. 
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To  overcome  the  second  objection,  if  the  value  of  U(l)  for  an  ordering  can  be 
expressed  in  terms  of  the  second  target,  we  can  redefine  value  by  interpreting  c  for  a  target  as 
its  utility  value  when  it  is  second. 

4.2  Value  for  Utility  One 


For  utility  1,  c  *  0,  therefore,  the  generic  value  algorithm  reduces  to 


B  +  R 
B  +  S 


smce 


Bi  +  Ri 

B^S 


B2  ^  ^2 

>  -  ifandonlyif  R^Bj  +  Rj)  >  R2(B2  +  R2). 

B2  +  S 


Therefore,  the  definition  of  a  value  for  utility  one  is 

VAL1  -  R(B  +  R). 

In  the  two  target  value,  the  order  of  battle  based  on  VAL 1  will  maximize  E  [  U  1  ] . 
4.3  Values  for  Utilities  Two  and  Three 

For  utility  2,  c  ■  1/2.  No  simplified  value  can  be  found  for  this  case. 


(19) 


VAL  2 


B(B  +  R) 


(B  +  S)(1/2R  +  B) 


(20) 


Orders  based  on  VAL  2  will  maximize  E  [  U  2  ] .  VAL  2  can  also  be  written  as 


B  f 


B  +  S 


R  j 

1  + - or 


R  +  2BJ 


2B 


R  +  2B 


1  + 


S-R 
B  +  R  ; 


but  neither  of  these  forms  seems  to 


lead  toward  the  elimination  of  S. 
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In  the  case  of  utility  3,  the  value  of  c  » ( 1-R)/S ,  therefore,  the  generic  target  value  can  be 
written  as  the  following  definition 


VAL3 


BS(B-t-R) 

(B  +  S)(R2«R  +  BS) 


No  equivalent  form  without  S  is  apparent. 


4.4  Value  for  Utility  Four 


(21) 


Given  two  targets  for  utility  four,  c  -  0.  Thus,  we  are  interested  in  minimizing  the  pro¬ 
bability  of  no  hits.  This  occurs  when  >  B2 . 


VAL4  -  B . 


(22) 


5.  Concluding  Remarks 

Each  of  the  values  examined  has  interesting  features.  If  the  desired  battle  objective  is  to 
remove  as  many  targets  as  possible,  then  VAL 1  appears  to  be  best.  If  the  goal  is  to  inflict  as 
much  damage  as  possible  on  the  enemy,  as  measured  by  U3,  then  VAL  3  seems  most 
appropriate.  Unfortunately,  VAL  4,  which  almost  always  gives  optimal  results  when  consider¬ 
ing  a  complete  victory,  does  not  perform  well  for  other  considerations. 

One  of  the  obvious  needs  is  a  method  for  acquiring  accurate  values  for  the  vulnerability 
(P)  and  threat  (R)  parameters.  These  values  not  only  depend  on  inherent  target  characteris¬ 
tics,  but  also  the  battlefield  conditions  and  the  missions  assigned  to  the  friendly  fire  unit.  Ini¬ 
tially,  the  literature  could  be  perused  for  probabilities  of  hit  and  kill.  One  promising  statisti¬ 
cal  approach  would  be  to  utilize  the  CART  software,  with  input  in  the  form  of  experimental 
data,  Emulated  data,  and  officers  judgments  [5]. 

Additional  conditions  for  optimality  of  U 1,  U  2,  and  U  3,  as  well  as  other  utility  func¬ 
tions,  should  be  developed. 

The  battle  scenario  is  rather  simplistic.  Indeed,  the  battle  may  be  criticized  since  it 
assumes  the  friendly  fire  unit  has  only  one  weapon  whose  removal  terminates  the  battle. 
More  sophisticated  simulations  should  be  developed  and  the  results  from  all  models  should 
be  compared. 

The  values  and  evaluation  criteria  presented  h  this  paper  may  be  used,  but  should  be 
regarded  only  as  an  interim  step  in  the  development  of  optimal  target  engagement  orderings. 
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A  Performance  Model  for  a  System 
Using  Range  and  Angle  of  Arrival  Information 
Andrew  Anderson  Thompson  III 
Ballistic  Research  Laboratory 


Abstract 

This  paper  develops  a  performance  model  for  a  system  using  AoA  and  range 
information  to  estimate  the  location  of  a  target.  The  ideas  used  to  derive  and  validate 
the  performance  model  are  presented,  and  then  a  procedure  for  evaluating  specific 
sensor  systems  is  discussed. 

1.  Introduction 

Many  sensor  systems  have  been  developed  using  angle  of  arrival  (AoA)  informa¬ 
tion  for  two  separate  receivers.  As  a  rule  of  thumb  the  error  associated  with  these  sys¬ 
tems  is  a  function  of  range  to  the  target  over  the  separation  of  the  two  receivers.  As 
this  ratio  becomes  large,  the  estimate  loses  its  value.  H.  Bruce  Wallace  of  the  Ballis¬ 
tic  Research  Laboratory  (BRL)  has  proposed  that  range  information  also  be  utilized 
in  order  to  have  a  worthwhile  target  location  estimate.  There  is  a  large  class  of  poten¬ 
tial  sensors  fitting  this  description.  These  can  be  classified  according  to  the  type  of 
electronic  processing  they  use  and  the  parameters  that  are  important  for  that  type  of 
processing.  The  goal  here  is  to  find  a  general  model  that  allows  for  various  levels  of 
detail  in  the  performance  evaluation  of  this  class  of  sensor  systems. 

This  paper  develops  a  performance  model  for  a  system  using  AoA  and  range 
information  to  estimate  the  location  of  a  target.  The  ideas  used  to  derive  and  validate 
the  performance  model  are  presented,  and  then  a  procedure  for  evaluating  specific 
sensor  systems  is  discussed. 

First  the  two  dimensional  case  is  examined  in  detail.  Then  the  results  of  applying 
the  same  ideas  to  the  three  dimensional  case  are  presented.  Both  models  were  vali¬ 
dated  through  the  use  of  a  simulation.  The  procedure  used  to  validate  the  three 
dimensional  model  is  presented.  The  next  two  sections  present  some  electronic 
models  from  the  literature  associated  with  the  accuracy  of  angle  of  arrival  and  range 
measurements.  Finally,  a  method  for  the  evaluation  of  a  particular  system  is  given. 

2.  XY  Covariance  Model 

In  this  section  a  performance  model  for  a  system  processing  one  range  measure¬ 
ment  and  one  AoA  measurement  is  presented.  The  measurements  are  from  a  polar 
coordinate  system  but  the  system  performance  is  to  be  in  the  Cartesian  coordinate 
system.  The  focus  of  the  discussion  is  on  how  the  measurement  errors  effect  target 
estimation  in  the  Cartesian  plane.  The  relationship  between  an  xy  location  and  an 
AoA  range  coordinate  is  straight  forward. 

x*Rcos0  +  xo  (1) 

y  ■  Rsin©  +  y0 

when  R  is  the  range  to  the  target  and  0  is  the  AoA  to  the  target.  For  the  rest  of  the 
discussion  we  assume  that  x0  »0  and  y0  *  0 . 
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A  common  approach  for  relating  measurement  errors  to  the  x-y  domain  is 
through  the  partial  derivatives  of  the  location  with  respect  to  the  measured  quantities 
(Reference  1  and  2).  In  the  x  direction  the  changes  caused  by  range  measurement  are 
AR  cos  ©.  Changes  caused  by  increasing  the  angle  move  x  closer  to  the  origin  by  the 
amount  R  sin  (A0)  sin(@).  In  deriving  the  following  model  several  assumptions  will 
be  made.  Afterwards  it  will  be  shown  that  the  assumptioas  are  statistically  reasonable. 
Perturbations  in  x  due  to  measurement  errors  are  described  by 

x  +  Ax  ■  (R  +  AR)  cos  (0  -f  A0). 

After  expanding  the  cosine  term  we  have: 

(R  +  AR)  cos  (0  +  A6;  '•  R  cos  ©  cos  A©  -  R  sin  0  sin  A0+ 

AR  cos  0  cos  A0  •  AR  sin  0  sin  A0. 

Since  A0  is  small  we  will  assume  cos  A0  *  1  also  since  A0  is  small  we  assume 
sin  A0  =  A0  and  finally  we  assume  A0  AR  *  0 .  So  we  now  have: 

(R  +  AR)  cos  (0  +  A0)  «  R  cos  0  +  AR  cos  0  *  RA0  sin  0. 

Recalling  that  x=Rcos0  we  have 

Ax  ~  AR  cos  0  -  R  A0  sin  0. 

By  a  similar  argument  it  can  be  shown  that 

Ay  «  A  R  sin  ©  +  RA©  cos  0. 

Note  taking  partials  of  both  the  x  and  y  values  results  in  the  same  set  of  equations,  and 
thus  those  terms  we  have  ignored  correspond  to  higher  order  differentials.  We  will 
assume  the  measurement  errors  have  the  following  properties 

E  (AR)  -  0 

E  (A0)  -  0 

E  (AR2)  -  <t2r  (2) 

E  (A62)  -  4 
E  (AR  A0)  -  0. 

Perturbations  of  x  and  y,  can  be  expressed  in  terms  of  measured  quantities  as 
Ax  *  AR  cos  ©  -  A0  R  sin  © 

Ay  ■  AR  sin  ©  +  A©  R  cos  0 

(Ax)2  -  (AR)2  cos2  0  +  (A©)2  R2  sin2  0  -  2  AR  A©  R  sin  ©  cos©  (3) 

(Ay)2  -  (AR)2  sin2  ©  +  (A0)2  R2  cos2  0  +  2  AR  A©  R  sin  0  cos  0 

Ax  Ay  »  AR2  sin  9  cos  0  -  A02  R2  sin  ©  cos  0  -  AR  A0  R  sin2  0  +  AR  A0  R  cos2  0 

Combining  Equations  2  and  3,  we  get  the  following  expressions  for  the  variance  and 
covariance  of  the  target  location. 

E  (Ax)  -  0 
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(4) 


E  (Ay)  -  0 

E  (Ax2)  -  o\  cos2  0  +  o\  R2  sin2  0 
E  (Ay2)  -  a\  sin2  0  +  a\  R2  cos2  © 

E  (Ax  Ay)  -  .5  [a\  sin  2  ©  -  cr\  R2  sin  2  ©) 

These  equations  describe  the  error  matrix  for  the  system  in  terms  of  the  XY  coordi¬ 
nate  system.  In  matrix  notation  the  covariance  matrix  of  the  estimated  position  would 


E  (Ax2)  E  (Ax  Ay) 

4  P°*°y 

E  (Ax  Ay)  E  (Ay2)  • 

■ 

/*°x°y  4 

By  rotating  the  coordinates  through  the  angle  a  defined  by 

2  E  (Ax  Ay) 

tan  2  a  ■ - 

E  (Ax2)  •  E  (Ay2) 

we  decouple  the  system  and  can  determine  the  major  and  minor  axis.  (This  is  derived 
in  Appendix  A). 

From  Appendix  A  we  have 


1 

Major  axis  «  —  j 


E  (Ax2)  +  E  (Ay2)  +  ((E  (Ax2)  •  E  (Ay)2)2  +  4  E  (Ax  Ay)2) 1/2 


Minor  axis 


(5) 


1_ 

2 


|  E  (Ax2)  +  E  (Ay2)  -  ( ( E  (Ax2)  -  E  (Ay)2  )2  +  4  E  (Ax  Ay)2)  1/2j 


The  following  equalities  were  used  to  rewrite  Equation  5 

E  (Ax2)  +  E  (Ay2)  -  aK  sin2  0  +  a\  R2  cos2  ©  +  cos2  0  +  <r2e  R2  sin2  0 
■  <t2r  (cos2  0  +  sin2  0)  +  o2b  R2  (cos2  0  +  sin2  0) 

-<72R  +  <72eR2 

E  (Ax2)  -  E  (Ay2)  ■  ct2r  (cos2  0  -  sin2  0)  +  a2e  R2  (sin2  ©  -  cos2  0) 

«  <t2r cos 20 - R2 cos  2 © 

E(Ax  Ay)  «  .5  <72R  sin  2  0  -  cr2e  R2  sin  2  0 
Thus  the  major  axis  and  minor  axis  are  defined  by 

“  ( ( <4  +  R2  4>  *  ( (A  +  <4  R2)2  cos2  2  0  +  (cr2R  -  a2e  R2)2  sin2  2  0) 1/2 ) 

((4 +  r24)  ±(4-r24» 
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The  final  simplification  of  the  above  equation  results  in  principle  components  of 
either  a  R  and  R  <7  e .  In  the  case  of  uncorrelated  measurment  errors,  the  axis  describ¬ 
ing  the  covariance  structure  are  directly  related  to  the  measurement  errors. 

For  the  two  dimensional  case,  performance  can  be  discussed  in  terms  of  x  and  y 
or  in  terms  of  principle  components.  Although  the  ideas  are  the  same  in  three  dimen¬ 
sions  the  relationships  are  more  complex  and  it  is  difficult  to  give  a  purely  tri¬ 
gonometric  explanation. 

3.  XYZ  Covariance  Model 

In  three  dimensions  a  covariance  model  can  be  derived  from  range,  azimuth 
angle,  and  elevation  measurements.  The  elevation  angle  0  is  measured  from  the  posi¬ 
tive  z  axis  and  the  azimuth  angle,  fa  is  measured  from  the  x  axis  toward  the  positive  y 
axis.  Using  the  same  simplification  arguments  as  for  the  XY  Covariance  Model  the 
following  set  of  measurement  error  transformation  equations  is  obtained. 

Ax  ■  ARsinQcos^  +  A0R  cos  ©  cos  fa  -  A^  R  sin  ©  sin  fa 

Ay  -  AR  sin  0  sin  fa  +  A0R  cos  ©  sin  fa  +  Afa  R  sin  ©  cos  fa 

Az  ■  AR  cos  0  -  A®  Rsin  © 

From  these  equations  the  covariance  model  can  be  found,  the  quantities  of  interest 
are  as  follows. 

<r2x  “  sto2  ©  cos2  fa  +  °2%  R2  cost2  ©  cos2  fa  +  R2  sin2  ©  sin2  fa 
<r2Y  «  <t2r  sin2  ©  sin2  fa  +  a2e  R2  cos2  ©  sin2  fa  +  R2  sin2  ©  cos2  fa 
a  z  ■  <r2R  cos2  ©  +  <r2e  R2  sin2  0 

aXY  “  °\  8^2  ®  s*n  fa  cos  fa  +  c2e  R2  cos2  ©  sin  faco&fa-  ct2^  R2  sin2  ©  sin  fa  cos  fa 

ax z  “  ^2r  ©  cos  0  cos  fa  -  <r20  R2  sin  0  cos  ©  cos  fa 

oyz  -  c2r  sin  ©  cos  ©  sin  fa  •  <72e  R2  sin  0  cos  0  sin  fa 

By  finding  the  eigenvalues  and  eigenvectors  associated  with  this  matrix,  the  principle 
components  and  orientation  can  be  found. 

4.  Model  Validation  Effort 

Both  the  XY  and  the  XYZ  models  were  compared  with  simulation  data  to  verify 
their  performance.  Using  a  Gaussian  random  number  generator,  errors  for  range  and 
azimuth  or  for  range,  azimuth,  and  elevation  were  generated.  These  errors  were 
added  to  the  true  values  and  then  the  position  was  calculated  from  the  corrupted 
values.  Using  ten  thousand  such  points,  the  covariance  of  the  target  position  was  cal¬ 
culated  and  then  compared  to  the  covariance  predicted  by  the  model.  The  code 
designed  to  perform  the  simulation  is  included  as  Appendix  B. 

As  a  first  test,  the  determinants  were  compared  to  see  if  they  were  in  agreement. 

A  test  based  on  the  asymptotic  distribution  of  the  sample  covariance  was  used  for  this. 

It  is 

dct(S) 

det(E)  ^ 
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is  distributed  as  N(0,2p) 


where 

p  is  the  dimension  of  the  matrix 
n  is  the  degrees  of  freedom 

In  this  case,  an  acceptance  based  on  a  determinant  is  not  conclusive.  These  tests  have 
no  sensitivity  to  the  orientation  of  the  covariance  structure,  and  will  not  detest  certain 
differences  in  the  magnitude  of  the  major  components  of  the  matrix.  For  example,  a 
diagonal  matrix  with  components  (1  ,1)  has  the  same  determinant  as  the  diagonal 
matrix  with  components  (.1,  10)  cr  the  matrix  with  row  one  elements  (5,  3)  and  row 
two  elements  (3, 2). 

The  statistical  properties  of  a  covariance  structure  are  described  by  the  Wishart 
distribution.  The  use  of  this  distribution  in  the  three  dimensional  case  leads  to  six 
independent  tests.  Each  one  checking  a  separate  component  of  the  covariance  matrix. 
Note  that  the  off-diagonal  terms  are  based  on  estimates  of  the  diagonal  terms  and 
thus  have  more  uncertainty  associated  with  them.  The  procedure  used  was  as  follows: 

1.  Calculate  the  error  structure  from  the  model. 

2.  Calculate  the  error  structure  from  the  simulated  data. 

3.  Find  the  normalizing  transformation  based  on  the  model. 

4.  Apply  this  to  the  result  of  step  2. 

5.  Test  the  resulting  matrix  to  see  if  it  is  statistically  equivalent  to  the 
identity  matrix. 

In  each  of  the  cases  investigated  the  model  and  the  simulated  data  produced  covari¬ 
ance  structures  that  were  statistically  the  same. 

5.  AoA  Errors 

The  two  models  included  are  based  entirely  on  thermal  noise  and  should  be  used 
as  the  best  case  situations.  When  additional  sources  of  error  are  modeled  it  is  usually 
correct  to  take  their  root  mean  square  with  the  error  due  to  thermal  noise.  Dr.  Alex¬ 
ander  in  (2)  gives  the  following  two  equations  for  relating  electronic  parameters  to  ae, 

For  pulsed  AoA  processing  the  thermal  noise  of  a  phase  interferometer  is  given 
by 

(360/2n)C 

2 II  f  d  cos  0  4  (S/N)int<3 

For  an  amplitude  monopulse  the  thermal  noise  is 

25.4  ©B 

fftll  “  7/7 

(S/Nintq) 

where, 

c  is  the  propagation  velocity  (M/S) 
f  is  the  RF  carrier  (Hz) 
d  is  the  spacing  between  the  receiving  antennas. 

©B  is  the  antenna  half  power  beam  width  (deg). 
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S/Nintg  is  the  integrator  output  of  signal  to  noise  ratio. 

<>.  Range  Errors 


Errors  in  range  depend  on  the  ability  to  measure  the  time  of  arrival  of  a  given 
pulse.  The  range  error  is  at  »  C/2  at  where  <rt  is  the  time  error.  The  time  error  is 
dependent  on  pulse  type,  the  following  are  suggested  by  Skolnik(3): 

For  a  rectangular  pulse 

1/2 

r  ^ 

where  r  is  the  pulse  width 
0  is  the  bandwidth 


<rt* 


U/JE/Nj 


E  is  the  received  signal  energy 
N0  is  the  noise  per  unit  bandwidth 


For  a  trapezoidal  pulse 

6E/N0  . 


1/2 


is  the  rise  and  fall  time  of  the  pulse 
is  the  direction  of  the  top  of  the  pulse 


For  a  triangular  pulse 

2T2 


vl2(2E/N0)l/2 


For  a  Gaussian  pulse  of  the  form  s  (t)  -  exp  (« 


1.384ta 


) 


1.18 


If  the  pulse  has  the  form 
then 


*/»(2E/N0),/2 

sin  (jt  0  r) 


npr 


vT 


»0<2E/NO)1''2 


Continuous  Wave  Error 

C 

<7r  ■  - 

4  7r  A  f  (2  E/N0)1/2 

where  A  f  is  the  difference  between  the  two  frequencies. 

The  specific  electronic  model  will  vary  depending  on  the  method  used  to  extract 
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information  from  the  signal.  The  models  included  above  only  represent  some  of  the 
common  techniques.  The  important  thing  is  that  they  relate  specific  electronic  or 
geometric  parameters  to  the  measurement  errors  associated  with  either  range  or 
angle  of  arrival. 

7.  Analysis  Method 

In  using  these  models  to  analyze  the  performance  of  a  system,  the  following  steps 
must  be  taken. 

1.  Choose  a  AoA  error  model  and  assign  values  to  the  parameter. 

2.  Choose  a  range  error  model  and  assign  values  to  the  parameters. 

3.  Use  the  range  and  AoA  errors  as  input  to  the  XYZ  location  model. 

By  following  this  procedure  at  a  number  of  different  points  a  system's  perfor¬ 
mance  can  be  presented  as  a  function  of  target  location. 

8.  Conclusion 

The  models  presented  herein  can  be  used  to  evaluate  the  performance  of  many 
range  •  angle  sensor  systems.  The  performance  can  be  based  on  specific  electronic 
parameters  such  as  frequency  or  pulse  shape  or  on  more  general  specifications  such  as 
three  degree  angle  with  five  percent  range  errors,  This  work  could  be  continued  by 
designing  a  software  package  that  includes  the  selection  of  the  possible  options. 

This  application  is  typical  of  the  error  analysis  approach  used  in  many  engineer¬ 
ing  studies.  In  this  case ,  the  extra  step  of  checking  the  statistical  validity  of  the  model 
was  included.  The  dominant  feature  of  this  approach  to  system  analysis  is  to  start  with 
the  measurement  errors  and  follow  them  as  they  propagate  through  the  system  and 
degrade  the  ideal  system  performance. 
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APPENDIX  A 

We  wish  to  find  the  angle  a  of  rotation  in  the  coordinate  system  to  decouple  the 
system. 

Ax  ■  Ax  cos  a  +  Ay  sin  a 
Ay  ■ -Ax  sin  a +■  Ay  cos  a 


In  terms  of  the  new  coordinate  system  we  have 
E  (Ax)  -  0 
E  (Ay)  ■  0 

E(Ax)  -  E  (Ax2)  cos2  a  +  2  E  (x)  (y)  sin  a  cos  a  +  E  (Ay2)  sin2  a 
■  E  (Ax2)  +  E  (Ay2)  -  E  (Ax2)  sin2  a  -  E  (Ay2)  cos2  a 
+  2  E  (Ax  Ay)  sin  a  cos  a 


Recall  that 

sin  a  cos  a 


sin  2  a 


1-  cos 2a 
sin  o  * - 


2  1  + cos  2  a 

cos  a  -  - 


then 


E  (Ax2)  -  “(E  (Ax2)  +  E  (Ay2))  +  -  [E  (Ax2  -  E  (Ay2)]  cos  2  a 
2  2 


+  E(AxAy)  sin  2a 


Similarly  it  can  be  shown  that 

E  (Ay2)  -  “  [e  (Ax2)  +  E  (Ay2)]  -  -( E  (Ay2)  -  E  (Ay2))  cos  2  a 
2  2 

-  E(AxAy)  sin2a 

E  (Ax  Ay)  »  -  E  (Ax2)  sin  a  cos  a  +  E  (Ax  Ay)  (cos2  a  -  sin2  a) 

+  E  (Ay2)  sin  a  cos  a 

-  —  [e  (Ay2)  -  E  (Ax2)]  sin  2  a  +  E  (Ax  Ay)  cos  2  a 
2 

The  covariance  term  will  be  zero  if 

2  E  (Ax  Ay) 

tan  2  a  ■ - 

E(Ax^-E(V) 


Note  that  for  this  angle 


sin  2  a  ■ 

cos  2a  ■ 


_ 2  E  (Ax  Ay) _ 

(4  E  (Ax  Ay)2  +  (E  (Ax2  -  E  (ty2)2)1/2 
_ E  (Ax2)  •  E  (Ay2) _ 

(4  E  (Ax  Ay)2  +  (E  (Ax2)  ■  E(Ay2))2)l/2 


Using  these  relations  it  can  be  shown  that  the  major  and  minor  axis  are  defined  by 


1/2  (E  (Ax2)  +  E  (Ay2)  ±  (E  (Ax2)  ■  E(Ay2)  +  4  E  (Ax  Ay)2)1/2) 
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APPENDIX  B 


#include  <math.h> 

#include  "ranvar.h" 

#include  "stat.h" 

maln() 

{ 

float  range, r,rjd,r__var,r_sq; 

float  theta,th,theta""sd,theta_var; 

float  psi,p,psijd,psl_yar; 

float  x,y,z^jum,yjwn,zjum,x_-var,yi-var,2-var; 

float  xsq,ysq,zsq,xyjxz,yz;“ 

float  r  sim,th_sim,psi  sim; 

float  slnth,co8th,8inpslico8psi,sin2th,cos2p9i,cos2th,sin2psi; 
float  cov_xy,cov_>xz,cov_yz; 

float  x2_mod,y2~mod,z2_modp(y_mod,xz_mod,yz_mod; 
float  data_det,mbdela.detiminu8,piu8lzJestlprob; " 

int  i,n,seed; 

/*  initialize  the  variables  for  this  run  */ 
range -50; 

rjd-. 05 ‘range;  /*  five  percent  range  */ 

r“var»r_sd*r_sd; 

r~8q- range ‘range; 

tReta-M_PI/4; 

thetajd«5*2*MJPI/360;  /‘five  degree  error  elevation  */ 

theta”var  -  theta  sd’theta  sd; 

psi-MPI/4; 

psi_sd-5*2‘M  PI/360;  /*  five  degree  error  azimuth  •/ 
psfvar  -  psl_sd‘psi_sd; 

n»  10000;  /*  number  of  replications  */ 

seed *23719;  /*  random  number  seed  •/ 

xsq-0; 

ysq-0; 

zaq-0; 

xjum-0; 

yjum-0; 

z_sum«0; 

xy-0; 

xz-0; 

yz-0; 

for  (i»0;i<n;l+  +) 
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{ 

/*  rvjauss  is  a  gaussian  random  number  generator 
the  first  two  sections  find  the  measured  value 
and  calculate  the  position 

7 

th  -  theta+theta_sd*rv jgauss(seed); 
r  -range + rjd*rvjauss(seed); 
p-psi+psijd*rv _gauss(seed); 

z»r*cos(th); 

x»r*sin(th)*cos(p); 

y«“r*sin(th)*sin(p); 

x_sum+  *x; 
y”sum+«y; 
z~sum+  "z; 

mm 

xsq+  -x*x; 
ysq+  «y*y; 
zsq+  »z*z; 
xy+  ■x’y; 
xz+  «x*z; 
yz+  -y*z; 

}  /*  end  of  replication  loop "/ 

x*xjum/n; 

y«y"‘sum/n; 

z-zjum/n; 

/•  the  following  values  are  the  covariance  elements  based  on 
the  simulations  data 

7 

x_var  -  (xsq-x*x_sum)/(n- 1); 
y”var  » (ysq-y  •y"sum)/(n- 1); 
z_var  -  (zsq--z*z_sum)/(n- 1); 
covjcy  *  (xy-x*y“sum)/(n- 1) ; 
cov “xz  ■  (xz-x*z”sum)/(n- 1); 
covj'z  *  (yz-y*z”sum)/(n*  1); 

/*  the  next  section  uses  the  model  to  find  the  predicted  covariance 
structure 

7 

sinth  » sin(theta); 

sin2th»sinth*sinth; 

costh-cos(theta); 

cos2th-costh*costh; 

sinpsi  ■■  sin(psi); 

sin2psi-sinpsi*sinpsi; 
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cospsi  =  cos(psi); 
cos2psi =cospsi*cospsi; 

x2_mod  «r_var*sin2th*cos2psi; 

x2~mod 4  -  theta_var*r_sq*cos2th*cos2psi; 

;;2_mod+  =psi_var*r_sq*sin2th*sin2psi; 

y2_mod = r_var*sin2th*sin2psi + theta_var*r_sq*cos2th*sin2psi, 
y2~mod  +  =psi__var*r_sq*'jin2th*cos2psi; 

z2_mod = r_var*cos2th + theta_var*r_sq*sin2th; 

xyjnod =rjrar*sin2th*cospsi*sinpsi + theta_var*r_sq*cos2th*<;ospsi*sinpsi; 
xy”mod-*psi_var*r_sq*sin2th*cospsi*sinpsi; 

xzjmod“r_var*sinth'*costh*cospsi-theta_var*r_sq*sinth*costh*cospsi; 

yz_mod»r_var*sinth*costh*sinpsi-theta_var*r_sq*sinth*costh*sinpsi; 

/•perform  determinate  test 

7 

plus  «  x  yar*y_var*z_var; 
plus  +  ■  cov_xy *  cov  _jyz  *  cov_xz  *  2; 
minus  ■  cov_xz  *  cov_xz  *  y _var ; 
minus  +  *  cov_xy  *  ccv_xy  *z_var ; 
minus  +  -  covjrz*covjrz*x_var; 
data_det  *  plus-minus; 

plus  *  x2_mod *y2_mod* z2_mod; 
plus  +  ■  2*xy_mod*xz_mod*yz_mod; 
minus  =  xy_m"od  *  xy_mod  41  z2_mod ; 
minus  +  =  y2_mod*xz_mod*xz_mod; 
minus  +  =  x2_mod*yzjnod*yz_mod; 
model_det = plus-minus; 

zjest  =  sqrt(n-l)  *  (datajiet/modeljJet  - 1); 
zjest  /*  sqrt(2*3); 
prob  =  cnf(z_test); 

printf("0); 

printf("**************** . . *********** . ***") 

printf("  Range  :  %f  theta :  %f  psi :  %P, range,  theta, psi); 
printf("Od(range)  =  %fsd(theta)=  %fsd(psi)»  %f rjd, theta jd, psi  jd); 
printf("Oimulated  Var(X)=  %f  model  value  was  %f',x_var,x2_mod); 
printf("Oimulated  Var(Y)=  %f  model  value  was  %f',y_var,y2_mod); 
printf("Oimulated  Var(Z)  =  %f  model  value  was  %f',z_var,z.2_mod); 
printf("Oimulated  Cov(XY)  =  %f  model  value  was  %f',cov_xy,xy_mod); 
printf("Oimulated  Cov(XZ)  =  %f  mode!  value  was  %f',cov_xz,xz_mod); 
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printf("Oimulated  Cov(YZ)«  %f  model  value  was  %f',cov>jz,yz_mod); 
printf("0  value  of  %i  with  probability  of  %f',z  test,prob); 

> 
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RELIABILITY  DESIGN  PROCEDURES  FOR  FLEXIBLE  PAVEMENTS 


Yu  T.  Chou 

Research  Civil  Engineer,  Pavement  Systems  Division 
Geotechnical  Laboratory 
US  Army  Engineer  Waterways  Experiment  Station 
Vicksburg,  MS  39180-6199 

ABSTRACT:  A  procedure  has  been  developed  to  analyze  lay¬ 
ered  elastic  flexible  pavement  systems  in  terms  of  reli¬ 
ability.  A  computer  program  RELIBISA  was  prepared  to 
carry  out  the  computations.  The  Rosenblueth's  method, 
instead  of  the  conventional  Taylor  series  expansion,  is 
used  to  estimate  the  expected  value  and  variance  of  the 
strains  (dependent  parameters)  based  on  the  input  mean 
values  of  independent  parameters,  i.e.,  aircraft  load, 
layer  thicknesses,  and  material  moduli.  The  relationships 
between  the  reliability  level  and  the  allowable  strain 
repetition  of  the  designed  system,  which  is  established 
with  results  computed  using  RELIBISA,  provide  a  decision¬ 
making  tool  for  engineers  to  design  pavements  at  a  desired 
reliability  level.  The  design  can  be  optimized  by  select¬ 
ing  thicknesses  of  the  bituminous  concrete  and  the  base 
layer  so  that  the  pavement  fails  in  fatigue  cracking  of 
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Preceding  Page  Blank 


Che  bituminous  concrete  and  In  subgrade  at  nearly  the  same 
traffic  level  for  a  given  reliability  level.  The 
reliability-e train  repetition  curves  have  a  steeper  slope 
with  the  bituminous  concrete  strain  failure  criterion  than 
with  the  subgrade  strain  failure  criterion.  This  steep¬ 
ness  Indicates  that  for  flexible  pavements  designed  using 
the  Corps  of  Engineer’s  failure  criteria,  the  design  has  a 
greater  degree  of  uncertainty  In  preventing  subgrade  fail¬ 
ure  than  fatigue  crocking  of  the  bituminous  concrete  sur¬ 
face  course.  However,  thie  may  not  be  true  In  real  cases 
because  the  bituminous  concrete  failure  criteria  are 
determined  baled  on  controlled  laboratory  test  data  which 
do  not  consider  the  uncertainties  existing  In  laboratory- 
to-fleld  correlations.  The  actual  performance  of  the 
pavement  with  respect  to  fatigue  cracking  will  be  more 
uncertain  than  is  considered  In  the  design.  The  sig¬ 
nificance  of  the  failure  criteria  employed  In  the  analysis 
on  the  derived  conclusions  is  discussed  and  Illustrated. 

It  was  found  that  the  performance  of  a  conventional 
flexible  pavement  Is  sensitive,  in  descending  order,  to 
variations  of  guar  load  P  ,  the  thickness  of  the  gran¬ 
ular  base  hg  ,  the  subgrade  modulus  Ej  ,  the  thickness 
and  modulus  of  the  bituminous  concrete  surface  course 
hj  ,  and  E1  ,  respectively,  for  the  subgrade  strain  fail¬ 
ure  criterion,  and  to  variations  of  P  ,  ,  Ej ,  , 

hj  ,  and  for  the  bituminous  concrete  strain  failure 
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criteria.  Although  pavement  performance  Is  more  sensitive 
to  the  variation  of  layer  thickness  than  to  variation  of 
tha  elastic  modulus  of  the  layer .  the  actual  variation  of 
material  moduli  In  the  field  le  known  to  be  much  larger 
than  the  variation  of  layer  thickness.  Strict  control 
during  construction  Is  recommended  to  reduce  the  degree  of 
material  variability  and  thus  to  lessen  the  degree  of 
uncertainty  and  to  increase  the  confidence  level  of  the 
designed  pavement. 


INTRODUCTION 


The  design  of  flexible  airfield  pavements  In  the  U8  Army  Corps  of 
Engineers  (USACE)  Is  currently  based  on  two  methods »  (1)  the  Cali¬ 

fornia  Bearing  Ratio  (CBR)  equation  that  Is  empirical  In  nature  and 
yields  a  design  thickness  for  a  given  design  condition,  and  (2)  the 
multilayered  elastic  method  that  Is  analytic  .1  In  nature  and  yields 
stresses,  strains,  and  deflections  In  the  pavement  system  for  a  par¬ 
ticular  loading  condition  and  pavement  geometry  which  In  turn  are  com¬ 
pared  with  established  fallurs  criteria  to  datermlne  the  performance 
of  the  given  pavement.  Both  design  approaches  are  deterministic, 
l.s,,  a  unique  pavement  system  is  designed  for  the  specific  set  of 
input  variables  necessary  to  solve  the  problem.  The  input  variables 
are  unique.  In  the  CBR  method,  a  pavement  thickness  Is  determined 
from  given  values  of  nubgtade  CBR.  gear  load  and  configuration,  tire 
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contact  araa,  and  design  coverage  level.  The  effect  of  material 
variability  on  pavement  performance  ia  considered  in  the  designer's 
selection  of  the  subgrads  CBR  value «  and  the  design  safety  factor  is 
implicitly  contained  within  construction  spec  if  lest  lone  such  as 
compaction  requirements. 

A  design  methodology  that  has  the  capability  of  considering 
design  parameter  variability  in  the  U8ACE  design  procedure  for  flexi¬ 
ble  airfield  pavements  using  the  elaetie  layered  method  is  presented 
in  this  paper.  The  design  procedure  is  expressed  in  probabilistic  and 
reliability  terms,  l.e,,  the  design  pavement  thicknesses  at  different 
performance  levels  are  computed  for  a  range  of  reliability  levels. 

The  designer  can  select  the  pavement  thickness  and  in  some  cases 
davslop  sn  overlay  design  scheme  based  on  the  desired  reliability 
level.  The  design  procedure  is  Incorporated  in  a  computer  program. 

By  using  the  procedure,  the  partial  effect  of  the  variability  of  each 
design  parameter  on  pavement  performance  can  also  be  investigated,  and 
its  effects  on  the  final  design  can  be  quantified.  Emphasis  can  be 
placed  on  the  crucial  parameters  to  be  tightly  controlled  in  the  con¬ 
struction  phases  and/or  the  crucial  loading  parameters  dictated  by  the 
Intended  use  of  the  pavement. 

PREVIOUS  WORKS 

Witczak,  Uzan,  and  Johnson  (1)  of  the  University  of  Maryland, 
under  a  contract  from  the  Waterways  Experiment  Station  (WES),  devel¬ 
oped  design  methodologies  for  rigid  airfield  pavements  in  terms  of 
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probability  and  reliability.  The  work  Involved  design  procedures 
based  on  the  Weatergaard  free  edge  stress  slab  theory  (2)  end  the 
multilayer  elastic  theory  (3).  Taylor  series  expansion  (4)  was  used 
In  the  probabilistic  analysis*  The  development  of  probabilistic  and 
reliability  design  methodologies  for  flexible  airfield  pavements  has 
been  conducted  at  WES.  Effort  was  first  conducted  on  the  CBR  design 
equation  (S)«  The  work  Is  briefly  described  in  the  next  paragraph. 

The  original  and  the  new  GBR  equation  for  flexible  airfield  pava- 
mente  are  shown  below  as  Equations  1  and  2.  respectively. 


where 

t  -  pavement  thickness,  in. 


a  -  a  traffic  factor 


F  ■  single-wheel  load  (or  the  equivalent  single-wheel 

loed  (ESWL)  In  the  case  of  the  multiple-wheel  loads) ,  lb 


CBR  *  California  Bearing  Ratio  of  the  subgrade  soil 


A  ■  tire  contact  area,  square  inches 
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Equation  1  was  formulated  In  the  1950' s,  and  Equation  2  was  formulated 
in  tha  early  1970's  based  on  additional  test  data  (6),  The  design 
.parameters  considered  were  the  load  P  (or  the  ESWL) ,  the  subgrada 
CBR,  the  tire  contact  area  A  ,  and  the  pavement  total  thickness  h  . 
The  expected  value  and  the  variance  of  the  dependent  variable  traffic 
factor  o  ware  estimated  using  the  Taylor  series  expansion  and  tha 
Rosenblueth  method  (7).  Differences  In  computed  results  between  the 
two  methods  were  found  to  be  small ,  although  the  derivation  of  the 
expressions  for  Taylor  series  expansion  is  very  complicated.  A  com¬ 
puter  program  was  developed  to  estimate  the  reliability  of  the 
designed  pavement  system  based  on  known  variabilities  of  design  param¬ 
eters.  Results  of  the  reliability  analysis  Indicate  that  prediction 
of  pavement  performance  Is  most  Influenced  by  variations  of  pavement 
total  thickness  and  la  least  Influenced  by  variations  of  tire  contact 
area  A  .  The  effects  of  variations  of  wheel  load  P  and  subgrade  CBR 
are  Identical.  Tha  relative  sensitivity  normalised  to  thickness  t 
for  parameters  t  ,  CBR  ,  P  ,  and  A  ,  in  general  cases ,  are 
approximately  1.  0.34.  0.34.  and  0.01,  respectively.  It  was  thus  con¬ 
cluded  that  In  the  future  analysis  of  pavements  involving  Input  param¬ 
eter  variabilities ,  the  affect  of  the  variation  of  wheel  contact  area 
may  be  neglected.  It  was  also  recommended  that  strict  quality  control 
be  exercised  during  construction  to  reduce  variations  of  pavement 
thickness  and  subgrade  CBR,  and  that  tha  Rosenblueth  method  (7)  be 
used  because  of  its  simplicity  and  accuracy  in  the  probabilistic 
analysis  of  layered  elastic  system. 
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PROBABILISTIC  AND  RELIABILITY  APPROACH 


In  analyzing  a  pavamant  structure  in  probabilistic  and  rail- 
ability  tarns*  tha  sxpactad  valua  and  varianca  of  a  function  (such  aa 
tha  computed  strasasa*  strain*  or  load  rapatltion)  should  first  ba 
datarnlnad*  and  tha  rallabillty  of  tha  daalgn  can  than  ba  avaluatad. 
Tha  Taylor  sarlas  axpanalon  (4)  and  tha  Rosanbluath  (7)  procadura  ara 
ganarally  us  ad.  Thaaa  uathods  ara  praaantad  balow. 


Taylor  Sarlaa  Expansion.  Tha  Taylor  formula  or  tha  axpanalon  of  a 
function  f(x)  ,  which  has  N  continuous  derivatlvas*  about  tha  func¬ 
tion's  naan  U  la 

f(x)  -  f(W)  +  f'<U)(x  -  U)  +1^1  (x  -  R)2 

+  ...  hlghar  ordar  tarns  +  ramalndar  0) 

Slnca  tha  sxpactad  valua  of  (x  -  R)  is  ssro  and  tha  axpacted  valua  of 
2 

(x  -  y)  la  tha  varianca  of  x  ,  l.a.,  E(x  -  y)  ■  0  and 
2  2 

E(x  y)  ■  °x  »  tha  sxpactad  valua  of  f (x)  bacomas 

E[f(x)]  *  f(y)  +  ■j  f"(y)o2  (4) 

Tha  varianca  of  a  function  f(x)  Is 

V[f(x)]  -  Etf2(x)j  -  E[f(x)]2  (5) 
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and 


Vtf<x)]#[f<v)l2c2  -  |  lf"<y)]20*  (6) 


Is  Equation*  4  and  6,  if  th«  random  variable*  can  b*  aaaumed 
normally  distributed,  th«  second-order  tarma  may  b*  neglected. 
Equation  6  also  aaaumaa  that  tha  varlabla*  art  unoorralatad. 


Roaanbluath  Mathod— Equation*  4  and  6  ara  obtalnad  from  th*  truncatad 
Taylor  aarla*  expanelon  of  th*  function  about  th*  axpactatlona  of  tha 
random  varlabla*,  Thla  mathod  requires  tha  axlatano*  and  continuity 
of  th*  flrat  and  aaeond  derivatives  of  tha  function.  Roaanbluath 
uaad  tha  point  estimate*  of  th*  function,  and  th*  axpraaalons  for  th* 
expected  value  arei 


E[fNl 

E[eN] 


for  on*  varlabla 


*t.“W  (.;♦.?) 

^ * 4. +  *-+  *  •"_) 

~s  +  *5+.  +  «!_  +  *!_+ +  *.++ +  *.h 


for  two  variables 


,  N  ,  , 

+  c  ,  +  c 

-•T 


? ") 


for  three  variablaa 


(7) 

(8) 

(9) 
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_r  N,  1/N  N  N  N  N 

E  6  ^  +  e+++-  +  £-H-+  +  e4+—  +  E+-++ 


N  .  N  .  N  ,  N 

+  <=+_+.  +  «+*+  +  «+ _ +  6 


+  *++  +  +  t"  ^.+  *  N 


+  e 


N 


(10) 


+  eN  .  +  «* 


-) 

V  M  H  y 


for  four  variables 

for  M  variables 


(ID 


Note  that  the  number  of  total  (arms  to  calculata  the  expected  value  of 
a  function  e  (strain  computed  using  the  elastic  layerad  BISAR  pro¬ 
gram  (8))  which  has  M  variables  is  2M  ,  and  N  has  a  value  of 
either  1  or  2  as  shown  In  Equation  5.  l.e.,  c  is  represented  by 
f(x)  ,  and  N  is  th«  power  of  the  function,  e+  and  c_  In  Equa¬ 
tion  7  are  the  strain  values  evaluated  at  the  mean  plus  one  standard 
deviation  of  the  variable  and  the  mean  minus  one  standard  deviation  of 
the  variable,  respectively.  c+_  in  Equation  8  Is  the  strain  value 
evaluated  at  the  mean  plus  one  standard  deviation  of  the  first  vari¬ 
able  and  the  mean  minus  one  standard  deviation  of  the  second  variable. 
Similar  reasoning  holds  true  for  the  other  terms. 


To  reduce  the  number  of  variables  in  elastic  layered  method  com¬ 
putations,  variations  of  Poisson's  ratio  of  pavement  materials  are 
neglected,  as  it  has  insignificant  affect  on  pavement  response  to 
loads.  The  variation  of  tire  contact  area  can  also  be  neglected  in 
this  computation,  based  on  the  conclusion  of  the  previous  study  of  CBR 
design  method  for  airfield  pavements  (5).  To  illustrate  the  use  of 
the  Rosanbluath  method,  the  computation  of  the  expected  value  of  the 
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strain  e  for  a  two-layer  flexible  pavement  Is  presented.  The  Inde¬ 
pendent  parameters  considered  but  uncertain  are  the  wheel  load  P  , 
the  elastic  modulus  and  thickness  of  the  top  layer  and  , 
respectively,  and  the  elastic  modulus  of  the  subgrade  E2  •  Other 
parameters  (wheel  contact  area  and  Poisson's  ratio)  are  precisely 
known,  l.e.,  the  standard  deviations  are  sero.  For  a  four-parameter 
problem,  Equation  10  is  used  to  determine  th'  expected  value  of  the 
strain  ®  .  Assuming  that  the  standard  deviations  of  the  parameters 
are  ap  ,  ,  aE2  •  »nd  0^  and  that  the  parameters  are  arranged 

in  the  order  of  P  ,  ,  E2  ,  and  h^  (l.e.,  the  order  of  the 

symbols  ++++  ,  -H  I—  ,  •*•,  etc.),  each  term  in  Equation  10  Is  computed 
using  the  BXSAR  program.  Once  the  mean  values  for  parameters  P  , 

,  Ig  ,  and  and  thslr  standard  deviations  cp  ,  oE1  ,  op2  ,  and 
are  specified,  the  expected  value  of  c  can  be  determined  from 
Equation  10,  and  the  variance  of  e  is  computed  using  Equation  5. 


Reliability  Analysis — as  soon  as  the  expected  value  and  the  variance 
of  a  function  (such  as  the  strain  values  computed  in  an  elastic 
layered  pavement  system  or  the  a  factor  In  Equation  2  representing 
the  traffic  performance  level)  are  determined,  the  reliability  level 
of  the  function  can  be  computed.  Reliability  is  defined  as  the  proba¬ 
bility  that  the  pavement  system  will  perform  its  intended  function 
over  its  design  life  (or  time)  and  under  ths  conditions  (or  environ¬ 
ment)  encountered  during  operation  (9) . 
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Two  failure  criteria  are  used  (10)..  The  criteria  for  allowable 
■train  repetitions  N  for  the  bituminous  concrete  is  mathematically 

expressed  as 


>W.u.(AC)  ' 10 


-A 


(12) 


where 


A  ■  5  Log10  e  4  2,665  log 


392 


c  ■  maximum  horisontal  tensile  strain  at  the  bottom  of  the 
asphaltic  concrete  layer 


Eac  ■  elastic  modulus  of  the  asphaltic  concrete ,  psl 

The  criteria  for  allowable  strain  repetition  N  for  the  subgrade  is 

expressed  as 

N.„  ....  -  200,00(/— i— 1  (1 

Allow*bl,(jubgrt(ta) 

where 

A  -  0.000247  +  0.000245  Log1Q  (Egubg) 


Esubg  "  eubgratU  modulus,  psi 

Gsubg  *"  "ubgrad*  strain,  dimensionless 


■  ■  °-06SS 
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The  bituminous  concrete  strain  criteria  in  Equation  12  are 
derived  based  on  laboratory  fatigue  data  on  the  breaking  stress  and 
strain  of  bituminous  base-course  materials  with  about  5  percent  air 
voids.  The  eubgrade  strain  criteria  In  Equation  13  are  derived  based 
on  the  full-scale  accelerated  traffic  test  data. 

The  strain  value  e  for  each  term  of  the  Rosanblueth  expression 
(Equations  7  to  11)  Is  computed  using  the  BXSAR  program.  The  perfor¬ 
mance  (strain  repetitions  to  failure)  of  the  pavement  may  be  estimated 
from  the  failure  criteria  shown  In  the  equations  for  two  different 
failure  modes. 

With  the  strain  value  t  assumed  normally  distributed,  the 
number  of  allowable  strain  repetitions  corresponding  to  e  +  eo(  (or 
e[l  +  C  •  CV(c) ) )  can  be  determined  from  Equations  12  and  13,  and  the 
probability  of  e  i  *[1  +  C  •  CV(c>]  Is  taken  from  the  normal 
distribution.  CV(c)  Is  the  coefficient  of  variation  of  c  ,  which  is 
the  ratio  of  the  standard  deviation  of  c  to  a  mean  of  c  ,  (l.e., 
c Jt),  and  C  Is  the  selected  number  varying  from  -3  to  +3.  The 
selection  of  C  values  less  than  -3  and  greater  than  +3  are  not 
necessary  because  the  areas  under  a  normal  distribution  curve  beyond 
-3  and  +3  standard  deviations  are  negligible.  The  computations  of  the 
reliabilities,  and  allowable  strain  repetitions  are  carried  out  In  the 
RELIBISA  Computer  program.  The  program  logic  of  RELIBISA  is  presented 
in  Reference  10.  Although  the  program  Is  prepared  based  on  failure 
criteria  presented  In  Equations  12  and  13,  other  failure  criteria  such 
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as  those  £or  highway  pavements  and  other  design  conditions  for  air¬ 
field  pavements  can  also  be  used  in  the  programs. 


ANALYSIS  OF  THE  FLEXIBLE  PAVEMENT 


In  the  analysis  of  an  airfield  pavement  in  terms  of  probability 
and  reliability,  the  RELIBISA  computer  program  is  used  to  calculate 
the  allowable  load  repetitions  for  a  given  pavement  section  for 
various  reliability  levels.  The  input  parameter  variations  are  CV 
(coefficient  of  variations)  defined  to  be  the  ratio  of  the  standard 
deviation  to  the  mean  value  of  the  parameter.  For  instance,  if  the 
mean  gear  load  is  178,000  lb  and  the  CV  of  the  gear  load  is  assumed 
to  be  10  percent,  the  standard  deviation  of  the  gear  load  will  be 
17,800  lb,  l.e.,  68.3  percent  of  the  time  the  gear  load  would  lie 
between  160,200  and  195,800  lb  (which  is  plus  and  minus  one  standard 
deviation  for  a  normally-distributed  variate) . 

Relationships  between  reliability  level  and  the  corresponding 
allowable  load  repetitions  are  established  for  many  pavement  sections 
with  various  input  parameter  variabilities.  The  reliability  of  the 
Corps  of  Engineers  flexible  pavement  design  model  is  0.5,  (.11),  but 
the  design  method  has  higher  reliability  value  because  of  the  design 
safety  factor  implicitly  contained  within  construction  specification 
such  as  compaction  requirements  and  the  selection  of  subgrade  CBR 
value . 
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Elastic  Moduli  of  Pavement  Layers — for  conventional  flexible  pave¬ 
ments  ,  the  critical  period  during  the  year  is  the  colder  winter  months 
,  for  the  surface  bituminous  layer  as  the  bituminous  concrete  becomes 
more  brittle  and  the  warmer  summer  months  for  the  subgrade  as  the  sur- 

e 

face  layer  becomes  less  stiffer.  Accordingly!  a  modulus  value  of 
1,000,000  psl  and  a  Poisson's  ratio  of  0.3  are  used  in  the  bituminous 
concrete  strain  criterion  (winter),  and  a  modulus  value  of  200,000  psl 
and  a  Poisson's  ratio  of  0.5  are  used  In  tha  subgrade  strain  criterion 
(summer) .  The  modulus  value  of  the  granular  base  material  is  assumed 
to  be  55,000  and  32,000  psl  for  the  bituminous  concrete  strain  crite¬ 
rion  and  the  subgrada  strain  criterion,  respectively.  The  selection 
of  these  values  la  explained  in  Reference  10, 


Analysis  of  a  Three-layer  Flexible  Pavement— a  gear  assembly  load  of 
178, 000- lb  B-747  aircraft  was  used  in  the  computation.  The  aircraft 
has  twln-tandem  gear  assemblies  and  the  wheels  are  spaced  44  by  58  In. 
Each  wheel  load  Is  44,530-lb,  and  the  tire  contact  pressure  Is 
182  psl. 

Two  series  of  computations  were  made  using  the  RELIB1SA  computer 
program  to  analyze  flexible  airfield  pavement  parameters  in  terms  of 
probability  and  reliability.  In  the  first  series,  the  thickness  of 
the  bituminous  concrete  surface  course  h^  is  held  9  in.,  and  a  range 
of  granular  base  layer  h2  is  assumed.  In  the  other  series,  the 
thickness  of  the  base  layer  h2  is  held  constant  30  in.,  and  a  range 
of  surface  layer  is  assumed.  In  both  computations,  the  CV  of 
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the  Input  parameters,  gear  loud  P  ,  moduli  of  the  bituminous  concrete 
E1  ,  the  granular  base  course  E2  and  the  subgrade  E3  ,  and  the 
thicknesses  of  the  surface  course  h^  and  the  base  course  h2  were 
assumed  to  be  0«1,  0.15,  0.2,  0.25,  0.1,  and  0,15,  respectively.  The 
computed  results  are  plotted  In  Figures  1  and  2  for  the  subgrade 
strain  criterion  (summer  temperatures)  and  the  bituminous  concrete 
strain  criterion  (winter  temperatures),  respectively. 

The  curves  shown  In  Figure  2  for  the  subgrade  strain  criterion 
are  generally  parallel  to  each  other,  except  in  the  area  where  the 
reliability  is  close  to  one  and  zero.  For  a  given  bituminous  concrete 
thickness  ,  increasing  the  thickness  of  granular  layer  h2  can 
increase  the  allowable  strain  repetition  of  the  pavement.  This  is 
aleo  true  if  this  procedure  is  reversed.  Figure  2  shows  that  for  the 
bituminous  concrete  strain  criterion,  the  performance  of  the  pavement 
can  certainly  be  improved  with  the  increase  of  the  thickness  of  the 
bituminous  concrete  surface  layer  hj  (for  a  given  thickness  of  the 
granular  layer).  This  is  also  true  if  the  thickness  of  the  surface 
layer  h^  is  held  a  constant  and  the  thickness  of  the  granular  layer 
h2  in  varied,  but  the  benefit  reduces  rapidly  for  very  thick  granular 
layer,  which  is  demonstrated  by  the  closely  spaced  curves  at  greater 
h2  values  presented  in  the  lower  part  of  Figure  2.  The  significance 
of  the  curves  is  that  for  a  given  pavement  thickness,  the  allowable 
strain  repetition  to  failure  varies  with  its  rsliability  level.  For 
a  9-in.  bituminous  concrete  surface  layer  hj  and  a  20-in.  granular 
layer,  the  allowable  strain  repetition  against  subgrade  failure 
(Figure  1)  at  a  reliability  level  of  0.5  is  8,500  strain  repetitions, 
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Figure  1.  Relationships  between  reliability  and  allowable  strain  repetition  for  flexible 

airfield  pavements  with  subgrade  strain  criterion 


i.e.,  the  chance  of  success  of  the  design  of  this  pavement  to  sustain 
8,500  strain  repetitions  of  the  B-747  aircraft  load  before  failure  is 
.50  percent.  Figure  1  shows  that  the  chance  of  success  can  be 
increased  to  80  percent  If  the  design  strain  repetition  is  reduced  to 
1,000.  If  the  8,500  repetitions  are  considered  as  a  20-year  design 
period  end  the  1,000  repetitions  are  thus  equivalent  to  2,4  years, 
there  Is  an  80  percent  chance  that  the  pavement  can  last  2.4  years 
without  failure  (with  routine  maintenance),  but  there  la  only  a 
50  percent  chance  that  the  pavement  can  last  a  full  20-year  design 
period. 

It  is  to  be  noted  that  the  flatter  the  slope  of  the  curves  In 
Figures  1  and  2,  the  greater  are  the  uncertainties  Involved  In  the 
design.  However,  the  ahapes  of  the  curves  are  influenced  by  the 
failure  criteria  (Equations  12  and  13)  employed  In  the  computations. 
This  will  be  discussed  later  in  this  paper.  The  curves  In  Figure  2 
for  the  bituminous  concrete  strain  criterion  have  steeper  slopes  than 
those  In  Figure  1  for  the  subgrade  strain  criterion.  Indicating  that, 
using  the  Corps  of  Engineers'  failure  criteria,  the  designed  pavement 
may  have  a  greater  degree  of  uncertainty  In  preventing  subgrade  fail¬ 
ure  than  In  preventing  fatigue  cracking  of  bituminous  concrete  surface 
course. 

For  a  given  design  strain  repetition,  the  relationships  between 
reliability  and  pavement  thickness  can  also  be  obtained  from  Figures  1 
and  2.  Engineers  can  choose  the  pavement  thickness  suitable  for  the 
selected  reliability  level  of  the  design.  This  point  can  best  be 

266 


266 


demonstrated  from  the  curves  presented  in  Figure  3  which  is  plotted 
from  Figures  1  and  2.  Figure  3  shows  the  relationships  between  the 
strain  repetition  and  layer  thicknesses  hj  and  h2  at  different 
reliability  levels.  The  relationships  in  Figure  3  can  be  helpful  to 
designers  in  selecting  the  allowable  strain  repetitions  of  a  given 
pavement  section  for  a  desired  reliability  level  or .to  vary  the  layer 
thicknesses  h^  and  h2  that  will  be  suitable  for  a  specific  design 
performance  level  of  the  pavement  for  a  given  reliability  level.  The 
slopes  of  the  curves  lndlcete  the  rate  of  change  of  allowable  strain 
repetition  due  to  change  of  layer  thicknesses  h^  or  h2  .  Obviously 
the  steeper  the  slope  of  the  curve,  the  better  the  design  for  that 
particular  failure  mode  would  be. 

The  curves  in  Figure  3b  in  the  region  where  the  granular  layer 
thickness  h2  is  less  than  30  in.  have  generally  the  same  slope  as 
those  in  Figure  3a.  Since  the  unit  cost  of  granular  base  course  is 
much  lees  than  the  bituminous  concrete  surface  course,  it  is 
economically  more  beneficial  to  Increase  the  thickness  of  the  granular 
base  course  (h2)  to  prevent  the  pavement  from  subgrade  failure.  This 
la  logical  from  the  structural  viewpoint,  as  the  base  course  is  placed 
directly  on  the  eubgrade.  However,  the  slopes  in  Figure  3b  drop 
slightly  at  greater  h2  thicknesses,  indicating  the  significance  of 
granular  layer  thickness  increase  also  drops  slightly  at  greater  h2 
thlckneso. 


267 


STRAIN  REPETITIOUS 


r 


MiUAMtirrv 


»oT 


liability 


id* 


h,  .  SO  IN. 


d 

COCdfICKNTOf  VARIATION 
M  t»  M,  I  l3  |  »,  I  H, 

oil  I  0.1ft  I  0.2  1  on  I  0.1  I  015 


10* 


P«  176.000  IB 
E,  •  200.000  P$i 


V<J 


»tv 


(SUMMER  TEMPERATURE  I 
E,  •  1,000.000  RSI 
.  (WINTER  TEMPERATURE  I 
l  j  •  33.000  PSi 
(SUMMER  TEMPERATURE  I 
I,  -  SS.OOOPSI 
(WINTER  TEMPERATURE1 


t,  -9.000  PSI 


i«’  L 

L 


J _ I _ l _ I _ I*’ 

14  TO  JO  «0  40 

h,.  INCHES 


1 _ I _ I _ I _ L 

4  10  JO  40  *0 

hf.  INCHES 


- , - 

SUflCRAOE  STRAIN  CRITERION 
(SUMMER  TEMPERATURE) 


J  'O’  1,1  1  .J 

M  0  14  70  SO 


J  10’  1  1  III  J 

40  0  10  JO  30  «0  40 


h„  INCHES  h*.  INCHES 


BITUMINOUS  CONCRETE  STRAIN  CRITERION 
(WINTER  TEMPERATURE) 


Figure  3.  Relationships  between  pavement  layer  thicknesses  and  strain 
repetition  (from  Figures  1  and  2)  with  flexible  pavements 


Figures  3c  and  3d  show  curves  plotted  for  the  bituminous  concrete 
•train  criterion.  The  stsep  slopes  In  Figure  3c  Indicate  the  struc¬ 
tural  benefit  of  Increasing  bituminous  concrete  thickness  to  prevent 
the  pavement  from  failure  of  surface  cracking.  The  flatter  slopes  of 
the  curves  In  Figure  3d  Indicate  relatively  little  significance  of 
base  course  support  In  a  flexible  pavement  In  the  limitation  of 
fatigue  cracking  of  the  surface  course.  Figure  3d  also  shows  that  the 
significance  of  base  course  reduces  rapidly  as  Its  thickness  continues 
to  Increase. 

The  curves  presented  In  Figure  3  provide  engineers  with  a  tool  to 
vary  the  layer  thicknesses  (h^  or  hg)  to  be  suitable  for  the  spe¬ 
cific  design  performance  level  of  the  pavement  and  for  a  given  reli¬ 
ability  level  of  the  design.  An  optimum  design  may  be  made  to  select 
the  thicknesses  of  the  bituminous  concrete  and  the  base  layers  so  that 
the  pavement  falls  In  fatigue  cracking  and  in  subgrade  at  nearly  the 
same  traffic  level  for  the  same  reliability  level. 

The  conclusions  drawn  from  Figures  1  to  3  are  based  on  a  subgrade 
modulus  of  9.000  pel.  Questions  arise  as  to  whether  a  stronger  sub¬ 
grade  support  of  the  pavement  would  reverse  the  observed  trend.  Com¬ 
putations  similar  to  those  presented  in  Figures  1  and  2  were  made  for 
a  subgrade  modulus  of  25,000  psi.  It  was  found  that  the  relationships 
between  the  reliability  and  the  strain  repetition  are  very  similar  to 
those  shown  In  Figures  1  and  2,  except  that  the  curves  shift  to  higher 
strain  repetition  values  because  of  stronger  subgrade  support.  This 
Is  more  predominate  In  the  subgrade  strain  failure  mode  than  In  the 
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bituminous  concrsts  failure  mode  because  stronger  subgrade  support  has 
a  greater  effect  on  pavement  performance  with  respect  to  subgrade 
failure  than  with  respect  to  bituminous  concrete  failure* 

Figures  1  and  2  show  the  relationships  between  the  reliability 
and  strain  repetition  for  a  glvan  aat  of  CV's.  Figure  4  ahowa  resulta 
of  the  effect  of  each  individual  paramatar  on  the  performance  of  flexi¬ 
ble  pavements.  Computations  were  made  to  vary  only  one  parameter  each 
time  while  variations  of  the  other  five  parameters  were  sero.  The 
results  presented  in  Figure  4  are  for  the  CV's  of  0.1.  The  figure 
shows  that  the  variabilities  of  different  input  parameters  have  dif¬ 
ferent  degree  of  Impact  on  pavement  performance.  For  both  subgrade 
and  bituminous  failure  criteria ,  the  pavement  performance  (allowable 
strain  repetition)  Is  most  sensitive  to  the  variation  of  the  aircraft 
load  P  .  For  the  aubgrade  strain  criteria,  the  pavement  performance 
Is  more  sensitive  to  variations  of  the  thickness  of  the  granular  base 
layer  h2  (which  is  placed  directly  on  the  subgrade)  and  the  modulus 
of  the  subgrade  E^  .  The  pavement  performance  Is  less  sensitive  to 
the  thickness  and  modulus  of  the  bituminous  concrete  surface  course 
hj  and  ,  respectively,  and  the  modulus  of  the  base  course  Eg  . 

For  the  bituminous  concrete  strain  criterion,  the  pavement  performance 
Is  more  sensitive  to  variations  of  the  thickness  and  modulus  of  the 
bituminous  concrete  layer  h^  and  ,  respectively,  and  is  less 
sensitive  to  variations  of  the  thickness  and  modulus  of  the  base  layer 
h^  and  Eg  ,  respectively,  and  the  modulus  of  the  subgrade  e3  .  It 
is  interesting  to  note  that  the  pavement  performance  is  more  sensitive 
to  variations  of  layer  thickness  than  the  elastic  modulus  of  the  layer 
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SUBGRADE  STRAIN  CRITERION 
(SUMMER  TEMPERATURE) 

E,  =  200,000  PSI,  E,  =  32,000  PSI 


material  In  both  failure  criteria,  and  It  la  leaet  sensitive  to  the 
variation  of  modulus  of  the  bituminous  concrete  surface  layer  E^  in 
the  eubgrade  strain  criterion  and  to  the  variation  of  subgrade  modulus 
Eg  In  the  bituminous  concrete  strain  criterion,  Thue,  the  perfor¬ 
mance  of  a  flexible  pavement  la  sensitive  in  the  descending  order,  to 
variations  of  P  ,  hg  ,  Eg  ,  h^  ,  Eg  ,  and  E^  In  the  subgrade 
strain  criterion,  and  to  variations  of  P  ,  ,  Eg  ,  h2  , 

and  Eg  in  the  bituminous  concrete  strain  criterion. 

The  significance  of  the  results  presented  in  Figure  4  may  also  be 
explained  from  another  viewpoint  by  using  the  values  listed  in 
Table  1,  Table  1  shows  the  ranges  of  computed  allowable  strain  repe¬ 
titions  within  +1  and  -1  standard  deviation  of  the  subgrade  strain 
value  for  six  different  cases.  Zn  each  case,  the  CV  of  onn  param¬ 
eter  la  equal  to  0,1,  and  the  CV's  of  the  other  five  parameters  are 
set  at  seto.  The  subgrade  strain  value  computed  for  the  pavements  Is 
0.0009404  in. /in.  Table  1  shows  that  the  standard  deviation  of  the 
subgradt  strain  is  the  largest  for  the  load  P  and  Is  the  smallest 
for  the  modulus  of  bituminous  concrete  surface  course  .  When  only 
the  varletlon  of  the  load  P  is  accounted  for,  there  is  a  68.3  per¬ 
cent  chance  (l.e.,  the  area  within  4-1  and  -1  standard  deviation  under 
a  normal  distribution  curve)  that  the  predicted  pavement  performance 
falls  within  the  range  of  56,160  to  479,000  strain  repetitions.  If 
only  the  variation  of  the  modulus  of  bituminous  concrete  E^  is 
accounted  for,  the  predicted  performance  for  the  same  percent  of 
chance  narrows  down  to  a  range  from  142,500  to  169,670  strain 
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Table  1.  Flexible  Pavement  Performance  Variations  as  Functions  of 
Variations  of  Input  Parameters  (Subgrade  Strain 


Coefficient 

Criterion) 

of  Variation 

Flexible  Pavement 

Standard 
Deviation 
of  8 train 

Strain 

Repetitions  for 

i. 

h_ 

Hi. 

O 

°e* 

c  +  c 

c 

t  - 

0.1 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0000940 

56)160 

479,000 

0.0 

0.1 

0.0 

0.0 

0.0 

0.0 

0.0000077 

142)500 

169,670 

0.0 

0.0 

0.1 

0.0 

0.0 

0.0 

0.0000249 

117)540 

207,120 

0.0 

0.0 

0.0 

0.1 

0.0 

0.0 

0.0000618 

78,790 

321,140 

0.0 

0.0 

0.0 

0.0 

0.1 

0.0 

0.0000330 

107,580 

227,570 

0.0 

0.0 

0.0 

0.0 

0.0 

0.1 

0.0000724 

70,360 

365,940 

*  The  subgrade  strain  computed  for  the  pavement  Is  0.0009494  in. /in. 


repetitions)  indicating  a  smaller  variation  and  thus  a  design  with 
less  uncertainty. 


VARIABILITY  OF  INPUT  PARAMETERS  OF  FLEXIBLE  AIRFIELD 

The  results  presented  in  Figure  4  are  based  on  the  analysis 
assuming  that  the  input  parameters  have  the  same  coefficient  of 
variation.  In  reality,  some  parameters  have  larger  variations  than  > 

others.  Although  the  pavement  performance  is  more  sensltiva  to  the 

% 

variations  of  layer  thickness  than  the  modulus  of  the  layer,  it  has 
been  found  that  the  thickness  variations  in  actual  field  constructions 
are  not  very  large;  the  average  CV's  are  generally  near  10  percent. 
(Nevertheless)  efforts  should  be  made  to  reduce  pavement  thickness 
variation  during  construction  as  much  as  possible.)  The  actual  varia- 
tlons  of  moduli  of  layer  materials  in  the  field  are  known  to  be  very 
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large.  The  CV  of  moduli  can  be  as  much  as  50  percent  or  more.  More 
efficient  construction  methods  and  equipments  should  be  used*  and 
.strict  compaction  and  quality  controls  should  be  exercised  in  con¬ 
struction  to  reduce  material  modulus  variations.  The  control  of  load 
variation  is  beyond  the  jurisdiction  of  pavement  engineers.  Since  the 
variation  of  aircraft  load  has  a  large  affect  on  pavement  performance* 
the  airfield  operators  should  be  informed  and  advised  to  limit  air¬ 
craft  overload  cases. 


SIGNIFICANCE  OF  FAILURE  CRITERIA 


All  the  reliability-strain  repetition  curves  shown  in  Figures  1 
to  4  have  steeper  slopes  for  the  bituminous  concrete  strain  criterion 
than  for  the  subgrade  strain  criterion.  It  seems  that  flexible  pave¬ 
ments  designed  using  the  Corps  of  Engineer's  failure  criteria  will 
have  a  greater  degree  of  uncertainty  In  preventing  subgrada  failure 
than  preventing  fatigue  cracking  of  bituminous  concrete  surface 
course.  However*  this  may  not  bo  true  for  pavements  in  the  field.  It 
Is  extremely  Important  to  point  out  that  the  subgrade  failure  criteria 
(Equation  13)  are  based  on  traffic  test  data  while  the  bituminous  con¬ 
crete  strain  criteria  (Equation  12)  are  derived  based  on  laboratory 
fatigue  data.  Slnca  failure  criteria  derived  from  laboratory  tests  do 
not  consider  the  uncertainties  existing  in  the  laboratory-to-fisld 
correlations*  the  actual  performance  of  the  pavement  will  be  more 
uncertain  than  is  considered  in  the  design.  Even  though  the  slopes  of 
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the  reliability-strain  repetition  curves  are  steeper  in  the  bituminous 
concrete  strain  criterion  than  In  the  subgrade  strain  criterion,  it  is 
not  necessarily  true  that  pavements  designed  using  the  Corps  of  Engi¬ 
neer's  failure  criteria  will  have  leaser  degree  of  uncertainty  In  pre¬ 
venting  fatigue  cracking  of  bituminous  concrete  surface  course  than 
preventing  subgrade  failure.  There  la  a  need  to  Incorporate  the  field 
uncertainties  Into  the  laboratory  determined  failure  criteria.  More 
discussion  on  the  significance  of  failure  criteria  on  the  reliability- 
strain  repetition  curves  (Figures  1  to  4)  Is  presented  In 
Reference  10. 


CONCLUSIONS 

Based  on  the  analysis  of  REL1BISA  computer  program  which  la  a 
layered  elastic  pavemant  design  approach  In  terms  of  probability  and 

-  i 

reliability,  the  following  conclusions  can  be  drawn  for  flexible  air¬ 
field  pavement  design  using  the  layered  elastic  method. 

The  relationships  between  reliability  end  strain  repetition  (such 
as  Figures  1  end  2)  can  be  used  to  design  a  pavement  in  terms  of  prob¬ 
ability  and  reliability.  For  a  desired  reliability  level,  the  thick¬ 
ness  of  the  bituminous  concrete  surface  course  or  the  thickness  of  the 
granular  base  can  be  varied  to  agree  with  the  designed  strain 
repetition,  or  the  allowable  strain  repetition  can  be  modified  for  a 
given  pavement  structure. 
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For  the  subgrade  strain  failure  criterion,  equal  changes  in  the 
thickness  of  bituminous  concrete  surface  course  or  granular  base 
. course  result  in  equal  changes  in  allowable  strain.  Since  the  unit 
cost  of  granular  layer  is  less  than  the  bituminous  concrete  course,  it 
is  economically  beneficial  to  Increase  the  thickness  of  the  granular 
base  to  prevent  the  pavement  from  subgrade  failure.  The  support  from 
base  course  has  relatively  lesser  significance  in  preventing  fatigue 
cracking  of  the  bituminous  concrete  surface  course  than  Increasing  the 
thickness  of  the  bituminous  concrete  layer  itself,  and  the  signifi¬ 
cance  reduces  rapidly  as  tha  thickness  of  the  granular  base  continues 
to  Increase  (Figure  3d). 

The  performance  of  a  flexible  pavement  is  sensitive  for  the  sub¬ 
grade  strain  criterion  to  variations  of  the  following  input  parameters 
(in  the  descending  order)  the  gear  load  F  ,  the  thickness  of  the 
granular  base  h2  ,  the  aubgrade  modulus  ,  the  thickness  of  the 
bituminous  concrete  surface  course  ,  the  granular  base  modulus 
E2  ,  and  the  modulus  of  the  bituminous  concrete  surface  course  , 
For  the  bituminous  concrete  strain  criterion  pavement  performance  is 
sensitive  to  variations  of  P  ,  h^  ,  Ej  ,  E2  ,  h2  ,  and  E^  . 
Although  pavement  performance  is  generally  more  sensitive  to  the  vari¬ 
ation  of  layer  thickness  than  to  that  of  the  elastic  modulus  of  the 
layer  material,  actual  variations  of  layer  thickness  in  the  field  are 
known  to  be  lesser  than  variations  of  material  moduli. 

The  results  of  the  reliability  analysis  are  very  much  influenced 
by  the  nature  of  the  failure  criteria  employed.  There  is  a  need  to 
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incorporate  the  field  uncertainties  into  the  laboratory  determined 
failure  criteria. 

The  relationships  between  reliability  and  strain  repetition 
developed  for  flexible  pavements  (Figures  1  to  4)  can  be  used  to 
optimise  the  design.  The  thicknesses  of  the  bituminous  concrete  end 
the  granular  layers  can  be  selected  so  that  the  pavement  Is  failed  in 
fatigue  cracking  of  the  bituminous  concrete  and  subgrade  failure  at 
nearly  the  same  traffic  level  and  the  same  reliability  level. 
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Nimble  Consultant's  Articles  of  Advice 


Field  of  64 


Before  first  visit  at  earlv  in  consulting_reladQnshiD 

1  Learn  what  you  can  about  the  subject  matter  in  client’s  field. 

2  Learn  what  you  can  about  client  and  client's  group. 

3  At  the  beginning  of  your  relationship  with  client,  agree  on  expectations;  what  will  be 

done,  by  when,  and  at  what  cost. 


fcuhCylatfy.dMring,  fust  g.Qn§.ultin  g_se§,si9.Q 

4  Find  out  what  client's  major  objective  is. 

5  Recognize  that  client  will  often  have  difficulty  articulating  major  objective. 

6  Find  out  specifically  what  data  are  available,  how  they  were  obtained,  and  what  are 

their  units. 

7  Determine  relation  between  particular  issue  under  discussion  and  overall  research  goals. 

8  Find  out  about  previous  studies  similar  to  the  one  under  discussion. 

9  Find  out  what  statistical  techniques  are  used  in  client’s  field. 

10  Learn  about  the  limits  and  constraints  on  client's  problem. 

1 1  Try  to  determine  what  client  knows. 

12  Try  to  determine  client's  view  of  statistics  and  statisticians. 

13  Formulate  goals  as  precisely  as  possible. 

14  Determine  who  real  decision-maker  is. 

15  Make  client  feel  comfortable;  avoid  scolding  and  excess  criticism. 

16  Be  prepared  to  spend  some  time  on  pleasantries. 

17  Reassure  a  frightened  client 

18  Avoid  any  adversarial  relationship  with  client 

19  Do  not  let  client  lead  you  up  the  garden  path. 

20  Do  not  feel  embarrassed  when  you  do  not  know  something. 

21  Encourage  client  to  expound  on  relevant  substantive  issues. 

22  Try  to  evolve  a  relationship  as  collaborative  equals. 

23  Have  a  clear  understanding  of  consulting  service  philosophy. 

24  Be  careful  of  the  'five-minute'  question. 
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Preceding  Page  Blank 


Purine  all  cons; lilting  sessions 

25  Where  possible,  make  client  feel  like  an  expert 

26  Rephrase  in  your  own  words  the  major  points  made  by  client.  (Do  this  often.) 

27  Encourage  cUent  to  rephrase  major  points. 

28  At  end  of  session,  clarify  in  writing  major  decisions  and  subsequent  goals  for 

consultant  and  client 

29  In  suggesting  solutions,  present  options  and  tradeoffs. 

30  Encourage  interchange  among  multiple  clients. 

31  Listen  carefully  to  the  throwaway  line. 

32  Be  aware  of  body  language. 

33  Be  friendly  and  patient. 

34  Take  good  notes. 

35  Have  your  homework  done  for  every  meeting. 

36  Ask  lots  of  questions. 

37  Make  sure  the  consulting  environment  is  pleasant. 

38  Be  careful  with  client-consultant  seating  arrangements. 

39  Avoid  excessive  interruption. 

40  Avoid  excessive  lecturing. 

41  Avoid  appearing  too  theoretical. 

42  Be  aware  of 'the  politics’  of  a  study  (if  such  exists). 

43  Make  the  statistical  procedure  fit  the  problem  rather  than  the  other  way  around. 

44  Encourage  pilot  studies. 

45  Probe  into  underlying  assumptions. 

46  Emphasize  exploratory  plots;  make  some  during  session  if  appropriate. 

47  Tailor  your  suggestions  to  the  capabilities  of  the  client 

48  Realize  a  major  role  of  consultant  is  helping  client  clarify  understanding. 

49  Do  not  hesitate  to  make  general  recommendations  to  general  experimental  effort 

50  Realize  you  can  always  make  a  contribution. 

51  Encourage  subsequent  visits  by  client  at  the  earliest  possible  stage  of  the  project. 

52  Visit  the  client's  home  turf  if  possible. 


While  working  on  problem  under  discussion 

53  Pay  attention  to  details  of  design  and  experiment  management 

54  Make  sure  data  and  computer  output  are  carefully  and  continually  scrutinized. 

55  Examine  carefully  time-order  of  data. 

56  Think  carefully  about  what  is  (are)  the  experimental  unit(s). 

57  Enumerate  sources  of  error. 

58  Dig  hard  on  diagnostics. 

59  Dig  into  literature,  when  necessary,  in  statistics  or  client's  field. 

60  Find  the  simplest  solution  that  does  the  job. 

61  Be  aware  of  political  reality. 

62  Be  prepared  to  make  some  compromises  with  the  real  world. 

63  Meet  all  deadlines;  approximate  solutions  on  time  usually  better  than  definitive  solution 

late. 

64  Make  written  reports  clear  with  major  points  emphasized,  with  lots  of  plots,  and  with 

minimum  of  excessive  technical  detail. 
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THE  SURVIVAL  PROBABILITY  FUNCTION  OF  A  TARGET  MOVING  ALONG  A 
STRAIGHT  LINE  IN  A  RANDOM  FIELD  OF  OBSCURING  ELEMENTS 


S.  Zacks  and  M.  Yadin 
State  University  of  New  York  at  Binghamton 
Binghamton,  NY  13901 


ABSTRACT.  A  target  is  moving  along  a  straight  line  path.  Random  portions  oi  the  path 
might  be  invisible  to  the  hunter  (in  shadow).  Shooting  trials  are  conducted  only  along 
the  visible  segments  of  the  path.  An  algorithm  for  the  numerical  determination  of  the 
survival  probability  of  the  target  is  developed.  This  algorithm  is  based  on  the  distribution 
of  shadow  length  which  is  also  developed. 
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1.  INTRODUCTION. 

The  present  study  is  focused  on  the  problem  of  determining  the  survival  probability  of 
a  moving  target,  which  is  under  attack  by  a  hunter.  The  target  (vehicle,  tank,  etc.)  is 
moving  along  a  straight  line  path,  which  is  partially  obscured  from  the  hunter  by  randomly 
distributed  objects  (trees,  clouds,  terrain  objects,  etc.).  The  target  can  be  destroyed  by  the 
hunter  only  along  the  visible  segments  of  the  path.  Visibility  contact  between  the  hunter 
and  the  target  is  needed  for  ro  time  units  for  a  shooting  trial  to  occur.  In  any  given  shooting 
trial  the  probability  that  the  target  is  destroyed  is  fixed,  If  the  target  survives  a  shooting 
trial,  another  identical  trial  may  be  attempted  if  continuous  visibility  for  ro  time  units  is 
possible.  If  the  target  enters  an  obscured  segment  of  the  path,  the  shooting  trials  terminate, 
until  visibility  contact  is  reestablished.  Under  the  above  assumptions,  if  the  target  has  to 
cross  a  visible  segment  of  length  L ,  its  survival  probability  can  be  approximated  by  the 
negative  exponential  function  exp{— qL) ,  for  suitably  chosen  constant  q,  0  <  q  <  oo.  The 
problem  is  that  the  number  of  visible  segments  on  the  moving  path,  between  two  specified 
points  Pj,  and  Py,  and  their  lengths  are  random  variables,  whose  distributions  depend 
on  the  characteristics  of  the  random  field. 

The  present  study  is  based  on  the  model  of  a  random  Poisson  field  of  obscuring  elements. 
This  model  is  presented  in  Section  2.  Under  the  assumptions  of  this  model,  it  is  relatively 
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simple  to  derive  the  conditional  distribution  of  the  length  of  a  visible  segment  on  the  right 
hand  side  (r.h.s.)  of  a  point,  P„,  on  the  path,  given  that  the  point  P*  is  visible.  This 
distribution  is  given  in  Section  3.1.  On  the  other  hand,  it  is  more  complicated  to  determine 
the  distribution  of  the  length  of  a  segment  which  is  obscured  (in  shadow).  In  Section  3.2 
we  present  the  methodology  for  determining  the  distribution  of  the  length  of  shadows. 
This  methodology  is  baaed  on  a  theory  given  by  Chernoff  and  Daly  (1957).  For  a  given 
point  P*  on  the  moving  path,  Chernoff  and  Daly  (C-D)  define  the  functional  T(x) ,  which 
is  the  right  hand  limit  of  the  shadow  to  the  r.h.s.  of  P* ,  cast  by  obscuring  elements  in 
the  field  which  intersect  the  ray  R*  from  the  origin  O  through  P* .  Employing  functions 
K±(x>y))  which  are  defined  in  Section  2  and  derived  explicitly  for  the  standard-uniform 
case  in  the  Appendix,  we  express  the  cumulative  distribution  function  (c.d.f.)  of  T(x) 
explicitly.  The  right  hand  limit  of  a  shadow  to  the  r.h.B.  of  P«,  is  U{x)  —  lim  Tn( a;), 
where  Tn+1(x)  =  T(Tn(x)),  for  n  =*  0, 1,  •  *  • ,  T°(x)  =  *.  The  relationship  between 
the  c.d.f.  of  Tn+l(x)  to  that  of  T"(x),  n  =  0, 1,  •  •  •  is  discussed  in  Section  3.2.  The 
distribution  of  U(x)  is  obtained  as  a  limit  of  that  of  Tn(x).  From  the  distribution  of  U(x) 
we  obtain  the  conditional  distribution  of  the  right  end  limit  of  a  shadow  to  the  r.h.s.  of 
P* ,  given  that  P*  is  the  first  point  in  the  shadow. 

In  Section  4  we  employ  the  results  of  Section  3  to  approximate  the  survival  probability 
function  S(x,y)  along  the  moving  path  between  the  points  PB  and  P„,  x  <  y.  The 
function  5(ar,y)  is  given  by  the  integral  equation 

5(®,y)  ==  A(*,y)  +  y  B(x,w)S(w,y)dw  (1.1) 

where  A(x}y)  and  B(x,y)  are  defined  in  terms  of  the  distributions  of  the  lengths  of 
visible  and  non-visible  random  segments,  as  shown  in  Section  4,  An  algorithm  for  the 
discrete  approximation  of  the  solution  of  (1.1)  is  given  in  Section  5.  Numerical  solutions 
based  on  this  algorithm  are  provided  there  too.  A  Quick  Basic  program  (version  4.5)  for 
computations  can  be  obtained  upon  request. 

In  a  previous  Technical  Report  [7]  we  approximated  the  survival  probabilities  by  deriving 
lower  and  upper  bounds  to  the  distribution  of  the  number  of  shooting  trials,  N ,  along 
the  path.  The  present  study  provides  the  method  of  computing  the  survival  probability 
function  S(x,  y) ,  which  is  required  for  various  applications.  With  the  new  algorithms  for 
determining  distributions  of  shadows  and  survival  functions  we  can  tackle  problems  like 
the  Hunter-Escort  problem,  which  will  be  discussed  in  another  paper. 
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2.  THE  RANDOM  FIELD  MODEL  AND  THE  DETERMINATION  OF  VIS¬ 
IBILITY  PROBABILITIES. 

In  the  present  paper  we  consider  a  two  dimensional  physical  model.  A  generalization  to 
a  three  dimensional  model  can  be  done  in  a  similar  fashion  to  that  of  Yadin  and  Zacks  [5]. 
The  moving  path  of  the  target  is  a  straight  line  C .  The  Hunter  is  located  at  a  point  O  (the 
origin),  at  distance  r  from  C.  Let  U  and  W  be  two  straight  lines  parallel  to  C,  located 
between  O  and  C,  at  distances  u  and  w  from  O,  respectively;  0  <  u  <  w  <  r.  The 
obscuring  objects  are  modeled  by  a  countable  number  of  disks  of  random  size,  which  are 
centered  at  random  points  in  the  ship  $,  bounded  by  U  and  W.  We  consider  a  cartesian 
coordinate  system  in  which  the  y-axis  is  a  straight  line  through  O,  perpendicular  to  C, 
which  intersects  W  and  C  at  the  point  (0,u),  (0,tt>)  and  (0,r),  respectively.  A  point 
P*  on  C  has  coordinates  (x,  r) , 

A  random  disk  is  represented  by  the  random  vector  (X,Y,  Z),  where  (AT,  Y)  are  the 
random  coordinates  of  the  center  of  the  disk,  and  Z  is  itB  random  radius.  Without  loss  of 
generality,  assume  that  the  sample  space  of  (XfY,Z)  is  S*  »  5  X  [a,  6],  where  0  <  a  < 
6  <  oo.  Let  B*  be  the  Borel  ff-fleld  on  S*.  Let  {(AT YJ, ,  t  ss  1, 2, •  •  • }  represent  a 
sequence  of  countable  random  disks  measurable  w.r.t.  the  same  space  (S*,0*,P),  It  is 
assumed  that  the  random  vectors  are  independent  and  identically  distributed  (i.i.d.),  and 
have  a  common  distribution  H(x,  y,  z) .  Let  F(z  j  x,y)  denote  the  conditional  e.d.f.  of  the 
radius  Z,  given  the  center  (Al,  Y)  is  at  (*,y).  Let  Ji(x,y)  be  the  joint  p.d.f.  of  (X,Y), 
such  that  h(x}y)  =  0  for  all  (x,y)  &  5.  We  further  assume  that  the  probability  that  a 
random  disk  intersects  either  O  or  C  is  zero.  Let  B  be  any  Borel  set  in  B* .  Let  N{B) 
designate  the  number  of  random  disks  with  coordinates  in  B . 

If  , 2?m}  is  any  finite  partition  of  S* ,  m  =  1,2, ••• ,  it  is  assumed  that  the 

random  variables  iV{Bi},  i  =  l,-**  ,m  are  independent,  having  Poisson  distributions 
with  expected  values 


=  xJJJ  dH(x,y,z), 
Bi 


i  =  1,' 


(2.1) 


0  <  A  <  oo.  Such  a  random  field  is  called  a  Poiaaon  field.  The  Poisson  field  is  called 
standard-uniform  if  dH(x}y^z)  =  hlc{x^y)f{z)dxdydz ,  where  0<h<oo,  C  is  a  subset 
of  S  which  represents  the  field  of  view  of  the  Hunter,  and  Ic(x,  y)  is  the  indicator  function 
of  C .  A  point  Pr  on  C  is  said  to  be  visible  from  O ,  if  the  ray  R*  from  O  through  Pt  is  not 
intersected  by  random  disks.  In  a  similar  manner  we  can  define  the  notion  of  simultaneous 
visibility  of  several  points  on  C.  In  our  previous  papers  [2,3,4]  we  have  introduced  the 
functions  K+(x,t)  and  K-(x,t)  for  0  <  t  <  oo;  where  A K±(x,t)  is  the  expected  number 


285 


of  disks  centered  in  5  between  the  rays  Re  and  R*±t,  which  do  not  intersect  R*.  Explicit 
formulae  for  K±(x,  t) ,  for  the  standard-uniform  case,  with  a  uniform  distribution  of  radii 
on  [a,&],  0  <  a  <  6,  is  given  in  the  appendix. 

Let  [L,  U]  be  an  interval  of  the  x  -coordinates  of  the  point  on  C  belonging  to  a  segment 
of  interest.  Let  L*  <  L  and  U*  >  U  be  properly  chosen,  and  C*  the  set  in  $  (trapez) 
between  the  rays  and  Rt/» .  One  can  verify  that  the  probability  that  P*  is  visible, 
for  some  L  <  x  <  U ,  is 

ij>(x)  a*  exp{-[p{C*}  —  -  L*)  —  A/f+(&,C7*  *)]}•  (2>2) 

For  a  formula  of  the  simultaneous  visibility  of  n  points  in  [L,  17],  see  Yadin  and  Zacks  [4]. 


3,  DISTRIBUTIONS  OF  LENGTH  OF  VISIBLE  AND  OF  SHADOWED 
SEGMENTS. 

3.1.  DISTRIBUTIONS  OF  THE  LENGTH  OF  VISIBLE  SEGMENTS. 

In  the  present  section  we  derive  a  formula  for  the  conditional  c.d.f.  of  the  length  of  a 
visible  segment  to  the  r.h.s.  of  P, ,  given  that  P*  is  visible. 

Let  I(x)  be  an  indicator  function  which  assumes  the  value  1  if  P*  is  visible,  and  the 
value  zero  otherwise. 

Let  L(x)  be  the  length  of  the  visible  segment  of  C  to  the  r.h.s.  of  P« ,  i.e., 

L(x)  st  inf{y  :  y  S>  *,  JJ  J(u)s*l}-®.  (3.1) 

*<»S» 

We  derive  here  the  formula  for 

V(l  |  x)  *  P{L(x)  <  1 1 I(x)  =  1}.  (8t2) 

=  1  -  P{L(x)  >  1 1 1(x)  ss  1}.  V 

Let  C *  be  the  set  of  (x}y)  points  in  5,  which  was  defined  in  the  previous  section.  We 
derive  the  formula  of  V(l  |  x) ,  for  L  <  x  <  C7 ,  and  0  <,  l  <  U  —  x . 

Let  C~(x)  be  the  set  bounded  by  Uy  W  and  the  rays  Rt*  and  R*.  Let  C(x,l )  be 
the  set  bounded  by  W,  W  and  the  rays  R* ,  R*+/|  and  C+(l  +  ®)  the  set  bounded  by  W, 
W,  R,+e  and  R u* .  Notice  that  C *  =  C.(x)  U  C(x,  l )  U  C+(l  +  x).  As  before,  we  denote 
by  /x{C}  the  expected  number  of  disks  having  centers  at  the  set  C,  as  given  by  (2.1). 
Accordingly, 

P{L( x)  >  /,/(*)  =  1}  =  exp{-[p{C_(x)}  -  XK-(xyX  -  L*)\  -  l*{C(x,l)} 

-  [n{C+(l  +  x)}-\K+{l  +  x,V*  -/-*)]} 

=  exp{-/i{C*}  +  A[K_(x,®  -  L')  +  K+(l  +  xtU*  -l-x)}}. 

(3.3) 
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Dividing  (3.3)  by  (2.2)  we  obtain 

P{L{x)  >  1 1  r(x)  «  1}  =  cxp{-A  [/<•+(*,  U*  -x)  - 1<+{1  +  x,U*-l-  *)]}.  (3.4) 


3.2.  THE  DISTRIBUTION  OF  SHADOW  LENGTH. 

We  have  denoted  by  U(x)  the  right  hand  limit  of  the  shadow  on  C  to  the  r.h.s.  of  PB . 
Let  D(u  |  x)  denote  the  conditional  c.d.f.  of  U(x),  given  that  the  shadow  starts  at  P* . 

Consider  the  rays  R,  and  R„  for  y  >  x.  Let  N(x,y)  denote  the  number  of  disks 
centered  in  5,  which  intersect  both  Rs  and  Rv.  Define  the  functional 

T(x)  «  sup{y  :  JV(®,  y)  >  1}.  (3.5) 


Furthermore, let  T'+1(x)  =  T^T^x)),  i  =*  0,1, •••  where  T°(x)  »  *.  Obviously,  Ti+1(x)  > 
2'*(x),  for  all  i  >  0,  and  therefore  U(x)  —  HmT^x).  U(x)  —  x  is  the  length  of  the  shadow 
to  the  r.h.s.  of  P* .  We  derive  first  the  c.d.f.  of  T(a) .  Clearly,  (T(x)  >  t)  s=  {N(x)  t)  >  1} . 
Thus, 

P{T(x )  <  <}  =  P{N(x,t)  =s  0}  *exp {-/i(<M)}>  (3.6) 

where  /i(x,t)  =  E{N(x)t)},  Furthermore, 

M(®,  t)  sa  ^{C*}  -  A K+(x,  U*  -  x)  -  AJM<,  t  -  L*)  +  Atf+(*>  *  -  ®) +  AJM<>  <  - *),  (3.7) 

where  t  is  the  coordinate  of  the  bisector  between  R*  and  R< .  Notice  that,  since  .FC^XjO)  « 
K-(x,0)  —  0  for  all  x, 

M*.x)  =  lim/*(x,<) 

‘i*  (3.8) 

=  /z{C*}  -  Atf+(x,u* -x)  -  \K-(x,x  -  L*). 

Hence, 

lim  P{T(x)  <  t)  as  t/>(x),  (3.9) 

which  is  the  probability  that  P*  is  visible.  Thus,  the  c.d.f.  of  T(x),  H(t\x)  is  zero  for 
t  <  x,  it  has  a  jump  point  at  x,  ff(x;x)  =  i/>(x),  and  is  absolutely  continuous  at  t  >  x. 
This  property  is  inherited  by  the  c.d.f.  of  TM(x),  Hn(t\x).  We  provide  now  the  recursive 
relationship  between  Hn{t\x)  and  Hn-\{t\x).  Introduce  the  bivariate  distribution  - 


a„{U,tr,x)  ==  Ffr-'W  <  h,T"(x)  <  h}. 
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Since  {T"{x)  <  t)  c  {Tn~\x)  <  t) , 


For  x  <  z  <  y  <  t, 


Hn(i;x)  =  P{Tn(x)<t} 

=  G„(f*,< ;*),  all  <*  >  t. 


(3.10) 


P{T"(*)  £  f  |  T”-a(»)  -  -  v)  =  «xp{-(M(v,<)  -  MM)|}-  (3.11) 


Indeed,  given  that  {Tn"a(*)  =  r,  Tn~l(n)  *  y) ,  (Tn(»)  >  t}  if  and  only  if,  there  exists 
at  least  one  disk  which  intersects  R,  and  Rt ,  but  doss  not  Intersect  R, .  Hence, 

Gn(Mai«)  -  j  J  exp{-[/i(ti,ta)-^(*,<a)]}rfGfn-i(*,tiia:).  (3.12) 

These  bivariate  c.d.f.  can  be  determined  recursively,  starting  with  Gi(ti,ta5®)  =  Hfax) 
for  all  <i  <  <a .  Moreover, 


Ga(*i,  ta;  *)  «  J*'  (J*  djy(u;  ®).  (3.13) 

Finally,  since  £Tn4-i(t;  *)  is  Hn(t\  ®)  for  each  t  >  ®  and  all  n  ■»  1,2,  •  •  •  the  c.d.f.  of  E/(*) 

is 

?M*)<i}«  Um  ffn(t}x)>  (3.14) 

W1  1  • 00 

Thus,  P{I7(®)  <  t}  =  0  for  all  t  <  *,  and  lim  P{U(x)  <  t)  =*  0(®).  The  conditional 

*1* 

c.d.f.  of  r/(®),  given  {/(®)  =  0}  is 


P{V(x)  <  1 1 1(x)  =  0}  = 


for  f  >  x 

1  -  0(x) 

0,  for  t  <  x, 


(3.15) 


We  are  interested,  however,  in  the  conditional  c.d.f.  D(u  |  x) ,  where  P*  is  the  first  point 
(the  left  hand  limit)  of  the  random  segment  in  shadow. 

Simple  geometric  considerations  yield  that  the  length  of  a  random  shadow  cast  by  a 
single  disk,  having  left  hand  limit  at  P* ,  with  center  on  a  line  parallel  to  U  at  distance  h 
from  O ,  and  disk  radius  Z ,  is 

U(xyh,Z)  =  r  tan  ^sin-1  +  tan-1  j  -  x,  (3.16) 
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where  (xc,  h)  are  the  coordinates  of  the  center  of  the  disk,  with 

*e  =  ?U  +  z(i  +  (2)  )  . 

(3.17) 

Thus,  if  a  <  Z  <  6  w.p.  1,  the  minimal  length  of  shadow  starting  at  x  is 

Gm(x)  es  r  tan  ^2  Bin'~1(B(x))  +  tan-^^)^ 

-  X 

(3.18) 

where 

1/3 

S(«)=  2.  [i+(£  +  2.(i+(i)J'),/JV 

w,  u»  yr  w  \  \rJJ  J 

u  m 

(3.19) 

In  Section  5  we  provide  computing  algorithms  for  the  numerical  determination  of  the 
c.d.f.’s  V(l  j  *)  and  D(u  \  x),  and  illustrate  them  with  a  numerical  example. 


4.  THE  SURVIVAL  PROBABILITY  FUNCTION. 

In  the  present  section  we  establish  the  integral  equation  (1.1).  Let  P*  and  Py  be  a 
visible  point  on  C  and  a  point  to  its  right,  L*  <  x  <  y  <  U* .  The  Hunter  starts  shooting 
trials  when  the  target  is  at  P* ,  The  attack  terminates  when  the  target  reaches  Pv ,  if 
it  has  not  been  destroyed  before.  Let  S(x}y)  designate  the  survival  probability  function. 
We  recognize  three  exclusive  and  exhaustive  events. 

(i)  The  visible  segment  to  the  r.h.s.  of  P*  terminates  at  a  point  to  the  right  of  Py ; 

(ii)  The  visible  segment  on  the  r.h.s.  of  P*  terminates  at  a  point  P < ,  t  <  y ,  and  the 
length  of  the  shadow  starting  at  P<  is  greater  than  y  -  t. 

(iii)  The  visible  segment  on  the  r.h.s,  of  P*  terminates  at  a  point  P< ,  t  <  y ,  and  the 
length  of  the  shadow  to  the  r.h.s.  of  P<  is  smaller  than  y  —  t. 

As  mentioned  in  Section  1,  the  conditional  survival  probability  of  a  target  moving  on  a 
visible  segment  of  length  L  is  exp{-g,  L) ,  for  some  0  <  q  <  oo.  Accordingly, 

5(«,  y)-e“«<»“*)(l-V(v-*|*)) 

+  fV  e“’,(,“*)(l  —  D(y  |  t))dV{t  —  *  |  x) 

J  X 

+  /V«(,-*>  J  r  S(z,y)D'(z  |  t)dz 
Jt  +  * 

where  D'(z  |  t)  =  |  t)  is  the  p.d.f.  of  D(z  |  t).  Notice  that  D'{z  \  t)  ■=  0  for  all 

t  <  z  <  um(t)  +  t,  Let  zm(t)  =  um(t)  +  t.  zm(t )  is  the  first  term  on  the  r.h.s.  of  (3.18) 
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with  x  =  t.  Let  tm(z)  be  the  inverse  of  z„,(t )  then,  by  changing  the  order  of  integration 
we  obtain 

rV 

S(z ,  y)D'(z  |  t)dz  V  dV(t  -  x  |  ®) 


ly-’tt. 

tv  (  /•<>»(*)-* 

-  jf  S(*’V)l/o 


x)|  dx. 


(4.2) 


(*  1 1  +  x)dV(t  | 

Jx  ^  JO 

Accordingly,  define 

A(»,  y)  -  -  V(y  -  *  |  *))  +  f  e“*‘(l  -  D(y  1 1  +  *)  W*  |  «),  (4.3) 

./o 


and 


£(*,*)  =  /  e-«‘I>'(*  |  <  +  aOdV(<  |  *). 

Jo 


(4.4) 


Thus,  the  integral  equation  (4.1)  can  be  written  as  in  (1.1). 


5.  ALGORITHMS  FOR  DISCRETE  APPROXIMATIONS  AND  NUMERI¬ 
CAL  EXAMPLES. 

In  the  present  section  we  consider  discrete  approximations  to  the  functions  Jfn(<;*), 
G,n(<l>toi®)j  n  >  2  and  5(*,y). 

For  a  given  integer,  JV,  partition  the  interval  (a;,y)  to  N  subintervals.  Accordingly,  let 
6  a  (y  -  x)/N)  to  =  x  and  tj  -to+j6)  j  =  0, 1,  •  •  •  ,  N . 

For  t  «  0,  •  •  •  ,  N ,  let 

&(»)  =  tf(<;;<o)  -  exp{-/i(t0,t,)}-  (5.1) 

For  t  ss  0,  •  •  •  ,  N  and  j  =  t,  •  •  •  ,  N ,  let 

< 

&(*>j)  =  Se3CP{-0*(<*»</)  “  MMi))}  •  [&(*)  1)],  (5.2) 

fc-0 

where  H\{- 1)  =  0.  This  is  an  approximation  to  (3.13),  Notice  that  £}%( 0,;)  =  H i(0)  for 
all  j'  =  0, 1,  •  •  •  ,  N ;  and  for  t  >  1 ,  j  >  i 

G2(i,j)  =  &i(0)  +  exp{^(t0,tj)}  •  5Zexp{-^(<fc,t>)}[^i(*)-  Hi(k  -  1)].  (5.3) 

*=i 


Moreover, 


Cj(t,  j)  =  Gx(hi)  for  all  i  >  j. 


(8.4) 


We  compute  afterwai  ds  recursively,  for  every  n  >  3 ,  i  =  !,•••  ,  JV ,  ,  N 


(5-5) 

*“  f  ~  1)  +  6n-l(fc  —  1,  /  —  1)], 

and  for  »  >  j 

6n(iJ)  =  6«(j,i).  (5.6) 

For  t  «  0,  drn(0,  j)  =  6„-i(0,j)  =  Hi(0),  j  =  0,-  •  • , JV.  After  computing  these  functions 
we  determine  #„(j)  =  $„(.?,  i)*  j  «*  0, 1,  •  •  •  ,  JV.  J-fn(j)  is  the  discrete  approximation  to 
the  c.d.f.  of  Tn( a:),  namely  Hn(t]x)]  i.e.,  Hn(tj\x)  «  #n(j). 

In  Table  5.1  we  present  numerical  results  obtained  by  applying  this  algorithm  to  the 
following  special  case. 

We  consider  a  standard-uniform  Poisson  field,  with  uniform  distribution  for  the  disk 
radius  on  the  interval  (a,  6).  In  the  appendix  we  present  the  functions  K±(x>t)>  t  >  0,  for 
this  case.  We  compute  the  numerical  example  for  Table  5.1  with  the  following  geometrical 
parameters:  r  =  100[m],  u  —  40[m],  w  =  60[m],  a  =••  l[m],  b  =  2.5[m],  x  =  10[m], 
L *  =  -100[m],  U*  =  100[m].  We  present  in  the  tables  the  values  of  #„(,?),  n  ~  1*2*3, 
j  =  0,  •  •  •  , 20 ,  when  6  =  l[m] . 

As  seen  in  Table  6.1,  the  convergence  of  &n{j)  to  the  c.d.f.  of  U(x)  is  quite  rapid.  We 
have  therefore  approximated  the  c.d.f.  D(u  |  x)  by  the  sequence  t)(j  \  i)  =  D(tj  i  *<), 
i  ss  0, 1, •  •  •  , JV ,  j  si,i  +  1, ••  • .  The  function  A(®, y)  was  computed  for  the  arguments 
U  *  i  by  the  approximation 

A(JV,JV)  =  1 

A(>N  -  1,  JV)  =  e“«4(l  -  ^(1  |  JV  -  1))  +  (1  -  D(N  |  JV  -  1)) 

A(N  -  j ,  JV)  =  -  VO  I  JV  -  j))  ?) 

+  £  «-*''■  i)8(V(J  |  JV  -  j)  -  V(l  - 1 1  JV  -  j))- 

Im  1 

-  (1  -  f>(JV  I  JV-i  +  J)),  i  =  2,  •  •  •  ,  JV 


where 


D(jv  |  jv  -  o  =  i(£(jv  |  jv  -  o  +  ^(JV  |  jv  - 1  + 1)). 

Z 


for  all  t  =  1,2,  •  •  •  ,  JV.  Recall  that  D(N  j  JV)  =  0  =  P(JV  |  JV  4-  1). 
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Table  5.1.  Values  of  H„U)  for  ^wo  va^uea  °f  A • 


A  =  0.02  [1/m2] 

A  =  0.2  [1/m2] 

j 

HxU) 

Hi) 

Hi) 

Hi) 

Hi) 

Ml) 

El 

0.4914 

0.4914 

0.4914 

0.0000 

0.0000 

0.0000 

1 

0.5434 

0.5434 

0.5434 

0.0000 

0.0000 

|M| 

2 

0.6009 

0.6008 

0.6008 

0.0000 

0.0000 

0.0000 

3 

0.6645 

0.6642 

0.6442 

0.0003 

0.0003 

0.0003 

D 

0.7342 

0.7336 

0.7336 

0.0021 

0.0020 

0.0020 

5 

0.8039 

0.8030 

0.8030 

0.0130 

0.0119 

0.0119 

6 

0.8664 

0.8652 

0.8652 

0.0576 

0.0522 

0.0522 

D 

0.9182 

0.9170 

0.1828 

0.1663 

0.1663 

8 

0.9569 

0.9559 

0.4164 

0.3843 

0.3842 

9 

0.9811 

0.9804 

0.9804 

0.6842 

0.6484 

0.6482 

10 

0.9934 

0.9930 

0.9930 

0.8783 

0.8507 

0.8505 

11 

0.9985 

0.9984 

0.9984 

0.9707 

0.9590 

0.9588 

12 

0.9999 

0.9999 

0.9999 

0.9981 

0.9958 

0.9957 

13 

1.0000 

1.0000 

1.0000 

HKES19H 

1.0000 

14 

1.0000 

1.0000 

1.0000 

1.0000 

1.0000 

1.0000 

15 

1.0000 

1.0000 

1.0000 

1.0000 

1.0000 

1.0000 

Similarly,  we  define 


&(N,N)  =  0 

&(N  -  1,  AT)  =  Ie“«*/2^(1  |  AT  -  l)£(AT  |  AT  -  1), 

« 

and  for  ;  =  2,  •  •  •  ,  Ar ,  /  =  1,  •  •  we  compute 

i 

&(N  -  j,#  -;  +  /)  =  £V?*(!-  *>[V(t  |  AT  -  ;)  -  £(*  -  1  |  N  -  ;)]• 

ini 

•  [D(N  -j  +  l\N~j  +  i)-  D(N  -  j  +  l~l\N-j-t  *)], 


(5.9) 


(5.10) 


where  D(N  -  j  +  1  -  1  \  N  -  j  +1)  =  0. 
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Table  5.2.  Survival  Probabilities  S(N  —  j,N ),  for  A  =  0.02(0.01)0.05, 
N  =  20;  a  =  2,  6  =  3.5,  r  =  100,  u  =  40,  w  =  60. 


AA 

0.02 

0.03 

0.04 

0.05 

0 

1.00000 

1.00000 

1.00000 

1.00000 

l 

0.81711 

0.82447 

0.83112 

0.83715 

2 

0.69730 

0.72040 

0.74052 

0.75808 

3 

0.61877 

0.65867 

0,69187 

0.71965 

4 

0.56729 

0.62203 

0.66572 

0.70094 

'  5 

0,53352 

0.60025 

0.65163 

0.69179 

6 

0.51135 

0.58728 

0.64401 

0,68729 

7 

0.49677 

0.57954 

0.63987 

0.68505 

8 

0.48718 

0.57489 

0.63759 

0.68390 

9 

0,47460 

0.56551 

0.63027 

0.67817 

10 

0,45820 

0.55111 

0.61788 

0.66784 

11 

0.43776 

0,53154 

0.60001 

0.65204 

12 

0.41376 

0.50726 

0.57671 

0.63029 

13 

0.38732 

0.47959 

0.54917 

0.60348 

14 

0,36009 

0.45064 

0.51979 

0,57418 

15 

0,33332 

0.42283 

0.49155 

0.54589 

16 

0.30991 

0.39803 

0.46678 

0.52137 

17 

G. 28902 

0.37709 

0.44652 

0.50189 

18 

0.27122 

0.35999 

0,43073 

0.48741 

19 

0.25614 

0.34607 

0.41851 

0.47687 

20 

0.24309 

0.33427 

0.40849 

0.46864 

Using  these  sequences,  we  compute 

S{N,N)  =  1 

S(N  -l,N)  =  A(N  -1,1V) 


and,  for  j  =  2,  •  •  •  ,  iV 

N-l 

5(JV  -  ;,  AT)  <=  A(N  -  ;,  AT)  +  B(N  -  ;,  Af)  +  ^  B(N  -  j,  i)S(i,  N).  (5.11) 

i=N-j+ 1 

The  function  S(x,y)  is  approximated  by  5(0,  iV).  In  Table  5.2  we  present  the  values  of 
S(N  —  j,  N) ,  for  the  geometrical  parameters  of  Table  5.1,  with  a  =  2 [m],  b  —  3.5[m]  and 
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several  values  of  A.  Also  here  5  =  l[m] .  The  value  of  q  is  —  fn(0.8),  This  corresponds  to 
the  situation  in  which  one  shooting  trial  takes  as  long  as  the  target  travels  l[m],  and  the 
probability  of  destroying  the  target  in  one  trial  is  0.8. 
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Appendix:  The  Functions  K±(x,t)  in  the  Standard- Uniform  Case. ,  With  Uniform  Dis¬ 
tribution  of  radii  on  (a,  6). 

Let  tf+(x,  t,  z)  denote  the  area  of  the  set  bounded  by  the  line  C+ ,  the  ray  Rc-h  t  t  >  0 , 
and  the  lines  U  and  W;  is  the  line  parallel  to  R*,  on  its  r.h.s.,  of  distance  z  from 
it.  This  is  the  set  of  all  disk  centers  between  R*  and  R*+<,  of  radius  Z  —  x,  which 
do  not  intersect  R*.  In  order  to  simplify  notation,  we  assume  that  w  =  r.  In  actual 
computations  we  substitute  xw/r  and  tw/r  for  x  and  t  in  the  formulae  given  below.  Let 
d  =  (x2  4-  w2)1/2 .  Simple  geometrical  considerations  yield: 


f  zd  1  T  w2  —  v.2  zd 

K+{xyt,z)  =  l{  ~<t4  - 1- 2— - 

{  t  JL2u>  u>  +  u 


+  / 


j  zd  \ 

|u  <  t  <  “j 


2  tw 


( tw  -  zd)7 


(A.1) 
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where  I{A}  is  the  indicator  set  function,  which  assumes  the  value  1  if  A  is  true,  and  the 

value  0  otherwise. 

Notice  that  JC+(x,  t,  z)  depends  on  x  only  via  x2 .  Symmetry  implies  that  K-(—x,  t,  z)  = 
K+(x,t,z)  =  K+(-x,t,z)  for  all  -oo  <  x  <  oo.  Hence,  /?+(&,<)  =  #■_(*,<)  and  we 
delete  the  ±  subscript  of  K.  Finally,  K(x,  t)  =  E{K(x,t)  Z)}  with  respect  to  the  uni¬ 
form  distribution  of  Z  over  (a,  6).  Let  x\  =  iu/d  and  x%  =  tw/d.  The  function  K(x)t) 
assumes  the  following  forms: 

(i)  If  b  <  xi , 

(ii)  If  a  <  xi  <  6  <>  xa 


♦  e(w  -•»*  -■»)  ■ 

(iii)  If  a  <  xi  <  xa  <  b 


(M) 


*  iii  -  *i)  *  S^s1-*  -  ■!>)  ■ 

(iv)  If  X!  <a  <b<x a , 

y(.,0.»-'(a+t)  +  *W +?+*>, 


(A.4) 


(-45) 


(v)  If  ®j  <  a  <  xa  <  6 

tw  X2  -  a 


K{x,t)  = 


d 


(xjjj  -  a2)  d- 


2  6  —  a  2(6  —  a) 


d3 


(®2~03)-  (A.6) 


6tu>(6  —  a) 


(vi)  If  ®2  <  a 


A'(®,  t)  =  0. 


(•4.7) 
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I.  INTRODUCTION 


A.  The  Role  of  Experiment  Design  in  Statistical 
Methodology 

Before  we  begin  to  study  graphical  methods  for 
experiment  design,  let's  review  the  role  of  thi% 
activity  in  the  overall  process  of  scientific 
investigation.  How  does  experiment  design.fifcin? 
This  figure  gives  Horace  Andrews'  view  of^er 
ursuit  of  knowledge  as  a  repeating  cycle(offoiTOihg: 
ypotheses,  designing  experiments,  colIecti$gdhtaV 
analyzing 'the  data,  leading  to  new  or  revised*: 
hypotheses.  Think  of  the  steepness  oftthifesjtefo  as 
tile-rate  of  knowledge  increase.  (Sources};  Andrews, 
1964).. 

The-.experiment  design  plays  a  key  rolfeifti 
determining  the  slope  of  the  line.  A  w.efrdesigned 
experiment  will  pro^oe; maximum  information  for  a 
given  level  of  efaorL  The  amount  of  information 
provided  by  the  experiment  or  data  collection  process 
can  be  measured!  in  several  ways.  Our  concern  as 
statisticians  v/iH  focus  on  three  measures: 

Variance 

Confounding 

Bias 


We'll  discuss  these  concepts  in  some  detail  or.  the  following  pages.  For  now,  we’ll  say  that  variance 
measures  the  precision  of  our  information,  confounding  the  ability  to  make  assertions  about  one 
hypothesis  mdeppident  of  the  verity  of  another,  and  bias  the  degree  to  which  our  estimates  measure  the 
things  we  think  they  measure. 


A  SIMPLE  STOCHASTIC  EXAMPLE: 

WEIGH  TWO  ITEMS  (A  and  B) 

SCALE  HAS  ERROR  WITH  variance  =  a 


APPROACH  1 


APPROACH! 


=A+B 

(C) 


A  4 

v«riince(A) 

A  2 

variance(B)  »<j 


*  2 
varimcc(A)  ■  _q 

(C*OV2  2 

variancc(B)  ■  $ 
(C-oyi 


Let's  illustrate  the  importance  of  experiment  design 
for  controlling  variance.  Most  of  our  data  collection 
efforts' wffll  have  to*  deal  with  random  perturbations  in 
the  values:  Reduced  variance  of  our  estimates  means 
greater  precision::  neater  information  for  the  same 
amount:  ofwork..  Let’s  weigh  two  items,  A  and  B 
with  a1,  scale'whose  output  contains  a  random 
component  with:  mean  zero  and  variance  o2  The 
first  picture  shows  one  design  to  estimate  the  weight 
of  A  and  B  in  two  weighingisessions.  The  resulting 
variance  of  the  weights  is  a2.  A  less  obvious 
weighing  design  is  shown  as  Approach  2.  This 
design  also  requires  only  two  weighing  sessions,  yet 
the  variance  of  the  estimates  has  been  reduced  by  a 
factor  of  two! 
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IKAWPLt. 


y-ao+  t,x,  **2x, 

®t-th-yi)/(xir*n)  “*i  «r((yi^Kyl*yi))  / ztx,,-*,,)  *»! 

ii«ovyi;  >  fxirxn)  ■»>  4i*K(y»«y4t*(yi*y»))  /  J(*ur*!i)  ■** 

BUT,  auppoie  model  wu: 

y-«o  +  i,xt  +i|X,  +  i|R,Xi 


In  tliis  example,  we'll  see  how  experiment  design 
can  be  used  to  reduce  bias.  Consider  a  set  of 
experiments  to  estimate  the  first  order  polynomial 
coefficients  for  a  Monte-Carlo  simulation  model 
output,  y,  which  is  a  function  of  two  parameters,  Xi 
ana 

The  first  design  requires  the  minimum  number  of 
runs.  The  estimated  values  for  the  coefficients  are 
random  variables,  since  the  simulation  model  output 
has  random  variation.  What  is  their  expected  value? 
If  the  simulation  response  truly  has  no  nonlinear 
component,  then  the  expected  value  (i.e.  long  run 
average  value)  of  the  estimated  parameters  will  be  the 
true  values. 

However,  if  the  true  response  has  some  nonlinear 
component,  as  illustrated  at  left,  the  first  design  will 
provide  biased  estimates  for  the  linear  coefficients. 
ITiat  is,  the  expected  value  of  the  coefficient 
estimates  will  not  be  the  true  values.  They  will  be 
offset  by  an  amount  that  depends  on  the  nonlinear 
terms. 


THEM 

•*!  ♦»)*1| 


The  second  design  requires  an  additional  run,  but  the  resulting  estimates  are  not  biased  by  the  presencebf 
the  interaction  term  if  it  is  present.  Planning  designs  to  minimize  bias  may  conflict  with  planning  a  design 
to  minimize  variance.  We  will  return  to  this  issue  later  in  our  discussions. 


•WE  EMPHA8IZE  THAT  THE  8ELECTI0N  OF  THE  MATRIX  OF 
EXPERIMENTAL  POINTS  REPRESENTS  ONLY  THE 
PROVERBIAL  TIP  OF  THE  ICEBERG.  THEREFORE. 

WE  STRESS  SUCH  MATTERS  AS  THE  NEED  FOR  CLEARLY 
DEFINING  THE  GOAL  OF  THE  TEST  PROGRAM, 
ENUMERATING  ALL  POSSIBLE  VARIABLES,  AND  HOW 
TO  HANDLE  THEM." 

■  Hahn  '  v. 

■WHAT  IS  THE  OBJECTIVE  OF  THIS  INVESTIGATION  T 
•J.S,  Hunter 


B.  Main  Steps  in  the  Design  of  an  Experiment 

This  concludes  our  brief  motivation  for  the 
importance  of  experiment  design  in  the  scientific 

Krocess.  Now  we  need  to  review  the.steps  that  must 
e  taken  to  produce  an  effective  experiment  design. 

Of  course,  our  whole  design  strategy  depends  on 
what  we  hope  to  learn  from  our  efforts.  The  quotes 
at  the  left  summarize  these  issues,  in  the  words  of 
two  well-known  applied  statisticians. 

Our  specific  interest  is  in  computer  simulation 
models.  The  table  on  the  next  page  is  a  classification 
of  common  goals  that  engineers  and  scientists  use 
simulation  models  to  achieve.  The  goals  are 
arranged  in  order  of  increasing  computational 
burden.  The  last  three  experimental  goals  apply  in 
design  and  policymaking  settings. 
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Goal  1:  Nominal  analysis  is  used  to  test  a  theory  or  to  validate  a  design.  Often  there  is  interest  in 
comparing  a  small  number  of  alternatives.  For  example,  a  pair  of  discrete  event  simulation  runs  might  be 
used  to  compare  the  performance  of  two  alternative  cugital  communications  protocols.  Here  the  vector 
function  F(x)  will  provide  various  measures  of  communications  performance  (e.g.  average  delay, 
maximum  delay,  delay  by  message  class,  etc.).  Schroer  and  Tseng  (1987)  use  a  DEDS  simulation  to 
perform  "what-if 1 1  analyses  on  alternative  manufacturing  system  parameters. 

Goal  2:  Sensitivity  analysis  serves  three  purposes.  First,  it  presents  a  local  model  of  the  system 
response  surface  (e.g.  a  Taylor  series  or  least  squares  polynomial  approximation)  that  can  be  used  to  study 
the  operating  behavior  of  the  true  system.  Second,  it  provides  an  opportunity  to  screen  out  unimportant 
factors  before  moving  on  to  more  detailed  experiments  (goals  3-5).  Third,  it  identifies  highly  influential 
variables  that  require  careful  control  for  process  stability  or  that  require  accurate  estimation  from  empirical 
data.  These  roles  are  interrelated. 

Goal  3:  One  might  want  to  design  a  particular  amplifier  circuit  to  minimize  power  dissipation  subject  to 
constraints  on  frequency  response,  output  power,  component  costs,  and  environmental  controls.  This 
optimization  could  be  quite  difficult  if  the  decision  vector  x  included  the  kind  of  circuit  components  used 
(e.g.  high  or  normal  efficiency  transformers)  and  if  the  response  included  random  perturbations.  It  would 
then  constitute  a  discrete  factor  stochastic  optimization  problem  with  implicitly  defined  constraints  (and 
objective  function).  The  satisfactory  general  solution  of  such  problems  is  beyond  the  scope  of  current 
methodology.  On  the  other  hand,  cases  where  all  variables  are  continuous  and  deterministic  have  been 
addressed  in  a  number  of  studies  (e.g.  Freeman,  et.  a!.,  1988). 

Goal  4:  Tolerance  analysis  identifies  the  multivariate  distribution  of  the  performance  vector, F(x ),  that 
will  occur  for  a  particular  (multivariate)  distribution  of  the  control  parameters  represented  by  x.  For 
example,  if  the  holes  in  an  electron  gun  grid  have  x,y  location  errors  with  a  multivariate  normal 

distribution  N  ( ( ° ),  <?i ) ,  what  will  be  the  distribution  of  the  spot  size  vertical  and  horizontal  diameter 
(DV,  DH)7 

Goal  5:  Tolerance  design  involves  trade-offs  between  incompatible  objectives.  The'optimal'  design  in 
terms  of  circuit  performance  may  result  in  a  design  that  is  difficult  to  manufacture.  Component  value 
variations  can  easily  move  the  operating  point  outside  some  set  of  constraints,  resulting  in  low  yield.  A 
more  manufacturable  design  might  be  the  point  would  allow  for  some  variation  in  component  values  and 
still  maintain  acceptable  (if  not  optimal)  performance.  These  design  problems  are  beyond  the  scope  of  this 
course. 
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EXPERIMENTAL  DESIGNS 


CLASSIFICATION! 


PILOT  /  SCREENING  /  EXPLANATORY  I  CONFIRMATORY 
SEQUENTIAL  /SIMULTANEOUS  (runipeci) 
CONTINUOUS  I  DISCRETE  (fKlon) 


Experiment  design  goals  are  classified  in  another 
dimension:  the  stage  of  the  investigation;  that  is, 
how  far  along  the  path  of  knowledge  we  have 
progressed.  At  the  earliest  stage,  a  pilot 
experiment  is  designed.  Its  main  purposes  are  to 
debug  the  experiment  running  and  data  collection 
procedure,  and  to  verify  the  Feasible  ranges  for  the 
independent  variables  (see  more  below). 

Screening  experiments  may  then  be  run  to 
eliminate  variables  with  little  effect  on  system 
performance.  This  allows  us  to  reduce  the  size  of 
Future  designs  to  study  the  important  effects  in  detail. 


:  yet  been  established.  Findings  from  i 
exploratory  designs  should  be  checked  with 
additional  data  From  a  confirmatory  experiment. 
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Exercise  1:  Identify  the  purpose  for  an  experiment  you  are  considering  in  vour  work. 

Give  the  purpose  in  words,  ana  identify  which  if  any  goals  in  the  previous 
table  correspond  to  your  interests.  Determine  whefher  you  are  at  the  pilot, 
screening,  exploratory,  or  confirmatory  stage. 
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The  second  step  in  the  experiment  design  process  is 
to  identify  variables  or  factors  that  will  concern  us  in 
the  course  of  this  investigation.  This  includes  things 
that  we  can  adjust  or  control  (independent  variables), 
resulting  system  outputs  or  performance  measures 
steps  in  experimental  design  (dependent  variables),  things  that  we  can't  control 

but  that  we  know  affect  the  system  (nuisance 
1  identify  variables). 

I  variables  These  three  kinds  of  variables  are  usually  all  that 

:  statisticians  identify  in  the  design  process,  but  there 

J  is  a  fourth  important  class:  intermediate 

variables.  Often  a  scientist  or  engineer  will 
understand  the  effect  of  one  factor  on  a  dependent 
variable,  but  will  not  be  able  to  control  that  factor 
directly. 


For  example,  the  strength  of  a  composite  material  may  depend  on  the  average  size  of  gas  bubbles  in  the 
material,  but  this  quantity  cannot  be  controlled  directfy.  Rather,  bubble  size  is  affected  by  mixing  rates, 
chemical  composition,  curing  temperatures,  etc.  One  of  the  most  difficult  tasks  in  the  initial  stages  of 
experiment  design  is  to  distinguish  the  intermediate  variables  from  the  truly  independent,  controllable 
variables. 


independent 

dependent 

NUISANCE 

INTERMEDIATE 


J  CLASSIFY  INDEPENDENT  VARIABLES 


WHICH  VARIED,  WHICH  FIXED 
WHICH  QUANTTrAnVE,  QUALITATIVE 
WHICH  EFFECTS  LINEAR,  NONLINEAR 


The  third  step  in  designing  an  experiment  is  where 
modeling  comes  into  play.  Usually  we  narrow  the 
scope  of  he  investigation  by  deciding  to  hold  some 
independent  variables  fixed  at  particular  values.  For 
the  factors  that  will  be  varied,  ranges  must  be 
established. 

We  usually  entertain  one  or  more  models  of  the 
system  response  as  a  function  of  the  independent 
variables  of  interest.  Often  this  will  be  a  regression 
model.  We  must  also  determine  which  variables  will 
be  treated  as  quantitative  (e.g.  maximum  vehicle 
speed,  total  number  of  troops),  and  which  as 
qualitative  (e.g.  kinds  of  vehicle,  type  of  weather). 
The  quantitative  factors  may  affect  the  response 
variable^)  in  linear  cr  nonlinear  ways. 

Systems  with  all  qualitative  factors  are  often  analyzed 
with  Analysis o/Variance (ANOVA) models. 
Systems  with  some  quantitative  and  some  qualitative 
independent  variables  are  often  analyzed  with 
Analysis  of  Covariance  (ANOCOVA)  models. 
Systems  with  all  quantitative  variables  are  often 
analyzed  with  Multivariate  Regression  models. 
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Exercise  2:  Classify  the  variables  for  an  experiment  that  you  propose  to  conduct: 

Independent  Variables: 

Quantitative: 

-Linear: _ 

-Nonlinear:  _ 

Qualitative:  _ 


Which  of  the  above  variables  will  be  held  fixed  for  the  proposed  experiment?  Which  will 
be  varied? 

FIXED:  at  what  values?  VARIED:  over  what  range? 

variable  value  variable  lower  limit  upper  limit 


Dependent  Variables: 

Quantitative  (preferred): 
Qualitative  (if  you  must): 

Nuisance/Noise  Variables: 


Intermediate  Variables: 
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ill  a  usrr  or  cofwnucnoN  methods 

llie  following  iimliotii  of  conuruciinj  factorial  tletlim 
literature!  ° 


(i)  Orthogonal  arrays. 

Of)  flulinccd  arrayt 

(j'h  Lxlin  iquaro  and  orthogonal  Latin  itiuam, 
<iv)  Hudamnrd  matrlcet. 

(v)  Pinfie  (tcomeirici. 

(vi)  Confounding 
(vli)  Qrwip  thaory. 

(Mil)  Algrtit»tod«ryuiip.i«liil»,, 

(ix)  Comblnaiorial  topology, 

(xi  Holdover, 

(xi)  Collnpiing  of  leveli: 

(xil)  Compoiiilon  (direct  product  and  direct  nun) 
(alii)  Codet, 

(xi v)  Block  daalgni. 

(xv)  r-aquxrex. 

(xvl)  Wel|ltlrt|  deitgni. 

(xvil)  Lattice  dceiint, 

(xviii)  Finite  rapid. 

(Mix)  One-et'a-tlme, 

(XX)  Trial  and  error. 


We  arc  now  ready  to  identify  the  set  of  conditions 
that  will  be  used  for  our  experiment.  This  is  the 
fourth  step  in  the  design  process,  and  the  main  focus 
of  this  report.  There  are  many  ways  to  choose  a 
design  for  the  experiment.  The  list  at  the  left  covers 
many  of  these,  both  graphical  and  non 'graphical 
(Raktoe,  Heydayat  and  Federer,  1981). 

Furthermore,  in  addition  to  the  considerations 
imposed  by  the  purpose,  the  number  and  kind  of 
variables,  and  the  model,  the  design  choice  will  also 
depend  on  whether  we  want  to  use  a  simultaneous 
or  sequential  strategy.  For  a  simultaneous  strategy 
we  establish  a  number  of  experimental  conditions  a- 
priori  and  collect  data  for  alt  conditions  before 
beginning  analysis.  This  has  advantages  where 
parallel  runs  are  possible,  such  as  in  agricultural 
experiments,  or  for  certain  parallel  computing 
applications  of  simulation  models. 

A  sequential  procedure  changes  the  design  based  on 
the  available  information  before  each  new  data  point 
is  collected.  The  settings  for  the  next  run  are  not 
determined  until  the  current  run's  results  have  been 
analyzed. 

In  this  course  we  well  focus  on  graphical  methods 
for  choosing  the  design  points,  we  will  devote  the 
greatest  attention  to  simultaneous  designs,  but  * 
evolutionary  operation  (EVOP)  sequential  designs 
will  be  discussed  also. 


4  CHOOSE  A  DESIGN) 

COOKBOOK 

MATHEMATICAL  (COED,  E-CHIP,  ACED) 
GRAPHICAL 


While  the  list  above  is  long,  the  realistic  strategies 
available  to  engineers  for  developing  an  experiment 
design  are  simple:  a)  choose  a  design  from  a  book, 
b)  use  a  computer  program  to  generate  a  design 
based  on  your  input  specifications,  or  c)  generate 
your  design  graphically. 
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Once  a  design  has  been  selected,  its  mathematical 
properties  should  be  checked  to  verify  that  it  will 
provide  information  which  meets  the  goals  for  low 
variance,  bias,  and  confounding.  Fortunately  and 
surprisingly,  it  is  usually  possible  to  assess  the 
quality  of  the  information  that  the  design  promises 
before  we  get  the  results!  We'll  discuss  the 
mathematics  of  this  procedure  for  the  General  Linear 
Model  in  a  later  section. 


GRAPHICAL  METHODS  FOR 
EXPERIMENT  DESIGN 


DESION  STEPS: 


1  PURPOSE 

2  IDENTIFY  VARIABLES 

3  CLASSIFY  VARIABLES 

4  CHOOSE  OR  CRB  ATE  A  DESION 
3  VALIDATE  THE  DESION 


WHY  GRAPHICAL  METHODS? 


These  five  steps  of  experiment  design  are 
summarized  in  the  table  at  the  left  Only  after 
completing  all  five  are  we  ready  to  carry  out  the 
experiment,  collect  data,  and  analyze  the  results. 


STANDARD  DUIONS  WONT  WORK 

EASIER  TO  CRBATI  NEW  DUIONS  WITH  AM  HT  I  RAIN  TOOLS 

INT1RRRSTATI0NS  SIMRUR 

CAN  11  USED  IN  DATA  ANALYSIS 


C.  Overview  of  this  Tutorial 

In  this  course  we  will  focus  on  steps  two,  four,  and 
five.  For  each  of  these  activities  we  will  present 
graphical  tools  to  make  these  tasks  easier  and  more 
funl  Only  step  5  will  we  deal  in  some  detail  with  the 
statistical  models  and  analyses  that  will  eventually  be 
performed. 

Each  new  technique  will  be  illustrated  with  one  or 
two  examples.  We  Introduce  the  main  example  in  the 
next  section. 

There  are  exercises  included  for  you  to  try  to  apply 
these  tools  to  a  design  problem  tnat  you  have  brought 
with  you. 

By  the  end  of  the  course  you  will  be  understanding 
and  using  graphical  tools  like  those  at  the  left  And 
remember,  an  experiment  that  is  well  designed  will 
provide  you  with  maximum  information,  and  will 
usually  be  easier  to  analyze! 
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D,  Introduction  of  Main  Example 

It  will  be  helpful  to  illustrate  these  methods  by 
considering  examples  along  the  way.  We  will  return 
often  to  one  particular  example  and  follow  it  through 
from  beginning  to  end.  It  is  based  on  a  real  study  of 
a  disk  pressing  operation. 

MAIN  EXAMPLE:  Disk  Pressing  Operation 

We  want  to  understand  a  vinyl  disc  pressing 
operadon,  and  through  this  understanding  improve 
the  quality  of  the  product  while  reducing  the 
manufacturing  costs.  The  disk  manufacturing 
operation  consists  of  a  number  of  steps, 

First,  vinyl  pellets  are  melted  and  formed  into  a  3" 
puck  by  the  extruder  shown  at  the  left.  The  puck  is 
removed  from  the  extruder  by  a  mechanical  arm  and 
placed  in  a  large  press.  The  vinyl  is  compressed 
between  two  steam-heated  molds  under  hydraulic 
pressure.  When  the  molds  nearly  touch,  a  switch 
triggers  the  end  of  the  steam  feed  to  the  molds  and, 
after  a  short  delay,  cooling  water  is  pumped  though 
the  same  internal  channels  in  the  molds.  The 
duration  of  this  cooling  oycle  is  controlled  by  a  tidier. 
After  this  time,  the  hydraulic  pressure  is  released,  the 
press  is  opened,  and  the  disk  is  removed. 

One  problem  with  disks  is  warp.  Warp  is  measured 
as  peak-to-peak  variations  in  the  height  of  the  outside 
edge  of  a  disk  as  it  is  spun  on  a  turntable.  Too  much 
warp  (more  than  15  mils,  say)  is  unacceptable.  We 
expect  that  warp  is  caused  by  internal  stresses  in  the 
dislc,  which  may  in  turn  be  caused  by  the  rate  at 
which  the  vinyl  is  compressed  to  form  the  disk  and 
the  cooling  process.  We  also  expect  that  thicker 
disks  will  oe  less  susceptible  to  warp. 


The  figures  at  the  left  show  typical  temperature, 
pressure,  and  thickness  profiles  over  a  single 
pressing  cycle. 
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Unfortunately,  warp  is  not  our  only  concern.  We 
also  must  worry  about  disk  weight.  Thicker,  heavier 
disks  contain  more  material,  ana  thus  cost  more.  We 
also  know  that  changing  timers  and  pressures  on  the 
press  will  change  the  time  it  takes  to  make  a  single 
disk.  Since  the  machines  which  make  the  disks  are 
expensive,  a  long  cycle  time  will  not  be  acceptable. 
As  in  many  experimental  situations,  we  have  multiple 
competing  objectives.  Increasing  the  disk  thickness 
may  decrease  warp  but  it  will  increase  disk  weight 
ana  consequently  material  costs  will  be  higher. 

We  will  return  to  this  example  to  design  an 
experiment  to  learn  how  the  pressing  parameters 
affect  warp,  disk  weight,  ana  cycle  time.  We  will 
want  to  predict  warp,  weight,  and  cycle  dme  as 
polynomial  functions  of  the  independent  variables. 
We  will  need  to  identify  the  key  variables,  choose 
models,  choose  a  design,  and  validate  the  design. 

(We  can  think  of  our  experiments  as  carried  out  on 
an  actual  disk  press  or  by  running  a  complex 
computer  simulation  of  the  disk  pressing  operation.) 
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II.  GRAPHICAL  METHODS  FOR  IDENTIFYING 
AND  CLASSIFYING  VARIABLES 

How  do  we  identify  and  classify  variables?  Without 
tools,  this  process  can  be  long  and  confusing. 
Exercise  2  was  intended  to  provide  a  worksheet  to 
help  in  this  process,  but  it  can  only  serve  as  a  list 
checker ,  not  a  list  generator. 


A.  Andrews  Diagrams 

One  graphical  tool  for  representing  the  factors  of  an 
experiment  is  the  Andrews  Diagram  shown  at  the 
left  (Andrews,  1964).  For  this  example.the  goal  is 
to  study  the  process  of  making  SPAM!  The 
incoming  arrows  represent  independent  and  nuisance 
factors,  the  outgoing  arrows  represent  the  dependent 
variables  of  interest:  flavor,  texture,  etc. 


This  graphical  tool  is  a  start,  but  it  fails  to  help  us  in  two  ways: 

1)  to  understand  the  distinction  between  independent  and  intermediate  variables,  and 

2)  to  represent  complex  causal  relations  involving  intermediate  variables,  where  a  response  variable 
depends,  through  a  chain  of  intermediate  variables,  on  one  or  more  independent  variables. 

The  Andrews  diagram  leaves  us  with  the  uncomfortable  feeling  that  we  haven't  identified  ALL  of  the 
important  factorstor  a  particular  experimental  situation,  The  next  two  tools  address  these  shortcomings. 


CONTROL FACTORS 


INPUT 

OPERATION 

OR 

PROCESS 
(A  VERB) 

OUTPUT 

s - 

MECHANISM 

OR 

ENABLING  RESOURCES 

IDEF  DIAGRAM:  Basic  Structure 


B.  IDEF  Diagrams 

These  causal  diagrams  were  first  presented  by 
Douglas  Ross  in 1977.  These  diagrams  can  6e  used 
to  describe  the  operation  of  any  complex  system. 
They  will  help  us  to  identify  all  the  important  factors 
in  the  system  we  arc  trying  to  model,  and  will  enable 
us  to  separate  independent,  nuisance,  intermediate, 
and  dependent  variables  easily. 

IDBF  diagrams  (also  known  as  SADT)  are  block 
diagrams  with  a  formal  process  for  drawing  and 
labeling  incoming  and  outgoing  arrows.  Arrows 
indicate  input  quantities  from  the  left,  control 

Earameters  from  the  top,  enabling  resources  from  the 
ottom  (mechanism)  and  outputs  to  the  right  The 
action  taking  place  is  described  in  the  box.  Thus  the 
arrows  are  labeled  with  nouns  (factors)  and  the  box 
with  verbs  (model). 
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TIMERS 

(SUSS  SWITCH 


TRIMMING  BLADE  OR  BIT 


These  figures  illustrate  the  use  of  IDEF  diagrams  for 
the  disk  pressing  example.  Notice  that 

a)  the  diagrams  are  hierarchical:  it  is  possible  to 
break  down  the  activities  in  a  high  level  box  to  lower 
level  boxes.  Outputs  from  one  lower  level  box 
which  are  inputs  to  another  lower  level  box  are 
intermediate  variables. 

b)  a  pattern  of  boxes  from  upper  right  to  lower  left 
indicates  a  dominance/dependence  relation. 


The  lower  figure  shows  a  more  detailed  breakdown 
of  the  pressing  activities.  This  allows  one  to  identify 
dependency  relationships  among  variables,  and  to 
uncover  variables  previously  overlooked.  For 
example,  we  must  now  wonder  whether  there  really 
are  no  control  factors  for  the  disk  trimming 
operation. 


Below  are  some  general  tips  on  using  these 
diagrams. 

1)  Break  down  to  the  bottom  level  of  detail  to 
identify  all  important  variables  and  classify  them  as 
independent,  intermediate,  etc. 

2)  Control  variables  (top)  are  the  'independent'  or 
'nuisance'  variables,  if  they  come  from  outside  the 
main  box,  otherwise  they  are  'intermediate' 
variables.  The  dependent  variables  exit  the  main  box 
at  the  right.  Distinguish  independent  variables  from 
nuisance  variables  by  whether  they  are  controllable 
(the  former)  or  not  (nuisance).  Controllable  means 
there  is  a  'knob'  to  adjust  the  parameter. 
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Exercise  3:  Draw  an  IDEF  diagram  for  r«  process  or  simulation  model  on  which  you 
plan  experimental  studies.  Carry  the  diagram  to  at  least  two  hierarchical 
levels  of  boxes,  three  levels  if  necessary.  Create  a  list  of  independent, 
intermediate,  nuisance,  and  dependent  variables  from  the  diagram. 
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MAC*«wes 


MAN  CAUSES 


C.  Ishikawa  Fishbone  Diagrams 

The  relation  between  independent,  intermediate,  and 
dependent  variables  is  made  clearer  in  an  IDEF 
diagram.  Another  graphical  tool  that  makes  this 
relationship  clear  is  the  Ishikawa  (1982)  Fishbone 
Diagram. 


ISHIKAWA  FISHBONE  DIAGRAM:  Basic  Structure 


AFTER  PRESS  WARP 


C  qtAQ«wTTCH<»carTo«) 


iw  oam  iwntN 

TMxaoi  amo  rrvM  cw 


These  diagrams  must  be  created  separately  for  each 
dependent  variable.  That  variable  is  named  in  a  box 
at  the  right  edge  of  the  paper.  A  horizontal  line  is 
extended  to  the  left,  and  aiagonal  lines  representing 
'causes’  of  the  dependent  variable  are  attached. 
Ishikawa  suggests  four  main  cause  lines:  people, 
machines,  material,  and  process.  Of  course  you  are 
free  to  choose  other  main  causes. 


The  diagram  at  the  left  illustrates  a  fishbone  diagram 
for  the  dependent  variable  POST-SOAK  WARJP  for 
the  disk  pressing  system.  The  diagrams  below 
shows  a  similar  fishbones  for  cycle  time  and  disk 
weighL  The  latter  two  both  occur  as  branches  of  the 
warp  fishbone  as  well  (but  are  not  elaborated  there). 
This  fishbone  interaction  is  common  for  multi¬ 
objective  studies. 

As  with  the  IDEF  diagrams,  it  is  possible  to  read  off 
the  independent,  dependent,  etc.  variables  from  the 
diagram. 

1)  Independent  and  nuisance  parameters  are  labeled 
causes  or  subcauses  which  themselves  have  no 
further  subcauses  (impinging  lines).  If  you  think  of 
the  diagram  as  a  tree,  these  are  like  the  leaves. 

2)  Intermediate  and  dependent  variables  are  causes  or 
subcauses  which  are  in  turn  caused  by  other  things. 
They  are  branches  with  twigs  or  leaves  attached. 
There  is  at  least  one  dependent  variable,  the  box  at 
the  right  of  the  diagram. 

3)  Independent  and  nuisance  variables  are 
distinguished  by  whether  a  'knob'  exists. 
Intermediate  and  dependent  variables  are 
distinguished  by  whether  or  not  you  are  directly 
concerned  about  the  parameter  value  (dependent),  or 
only  about  its  consequential  effect  on  other  measures 
of  system  performance  (intermediate). 


1  ots*  weiom  I 


T**  «rTY>*fK<XA» 
mo  r 
CWMMl 


The  circled  parameters  indicate  the  set  that  has  been 
chosen  to  vary  in  a  hypothetical  experiment.  These 
figures  are  from  Young,  et.  al.  (1987). 
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Exercise  4:  Draw  a  fishbone  diagram  for  one  dependent  variable  for  a  process  or 
simulation  model  on  which  you  plari  experimental  studies.  Carry  the 
diagram  to  at  least  two  hierarchical  levels  of  subcauses,  more  if  necessary. 
Create  a  list  of  independent,  intermediate,  nuisance,  and  dependent 
variables  from  the  diagram. 
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The  block  diagram  (not  IDEF!)  above  illustrates  the 
key  steps  in  a  printed  circuit  board  manufacturing 
operation.  Assume  we  have  built  a  discrete  event 
computer  simulation  model  of  this  operation  which 
we  will  use  to  optimize  the  throughput 


ISHIKAWA  FISHBONE  DIAGRAM:  Circuit  Board  Manufacturing 


This  diagram  shows  a  partially  completed  fishbone 
diagram  for  the  printed  circuit  board  process.  The 
main  causes  have  been  chosen  to  correspond  to 
processes.  The  variables  that  have  been  identified  so 
far  are: 

Independent: 

hole  diameter  specification 

sequencer  bin  filling  procedure 

insertion  arm  routing  instructions 

(’register  marks'  needs  more  specification  before  we 

have  the  appropriate  ’knobs')  - 

Dependent: 

throughput 

Intermediate: 

%  failed  insertions 
#  missing  part  occurrences 
hole  diameter 

Nuisance: 

optical  device  accuracy 
bent  leads 
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VARIABLES  FOR  THE  DISK  PRESSING  EXAMPLE: 

INDEPENDENT  VARIABLES: 

Timers  T-6.T-1 2,  T-13,  T-22 

Full  Clamp  Pressure 

Lo-HI  Transition  Pressure  Threshold 

Ram  Velocity 

Glass  Switch  Setting 

Vinyl  Composition 

Soak  Bath  Temperature 

Soak  Bath  Time 

INTERMEDIATE  VARIABLES  (some  of  them): 

Time  to  Reach  1 2"  Diameter 
Bead-Hub  Delta 

Disk  Temperature  at  Release  From  Press 


D.  Discussion  of  Main  Example 

The  IDEP  and  Fishbone  diagrams  have  helped  us  to 

Perform  Step  2  in  the  experiment  design  process, 
or  our  disk  pressing  example,  we  have  identified 
the  variables  shown  at  the  left.  This  list  tuts  been 
shortened  and  simplified  for  this  course  presentation. 


Step  2  in  experiment  design  involves  identifying  the 
key  variables.  In  Step  3  these  variables  are  classified 
as  fixed  or  varied,  linear  or  nonlinear,  quantitative  or 
qualitative.  All  or  these  decisions  are  intimately 
related  to  the  mathematical  model(s)  that  will  be  fitted 
to  the  system  response(s). 


NUISANCE  VARIABLES  (tom*  of  thorn); 
Room  Temperature 
Maximum  Steam  Pressure 

DEPENDENT  VARIABLES: 

Post-Soak  Warp 
Cyde  Time 
Disk  Weight 


OPERATtONAlAODIUNO  OIOISIONS: 

STIf  3  W IXAIRIMINT  OISION 

•  SELECT  A  SIT  OF  DIPIN  DINT  VARIAIlll  POP  STUDV 

•  CLASSIFY  THE  INDIPINDINT  VARIAIlll: 

WHICH  VARIED  IN  EXPERIMENT  WHICH  PIXIfi 

(SPECIFY  HANOI)  (AT  WHAT  VALUE) 

WHICH  QUANTITATIVE  VWICH  QUALITATIVE 

WHICH  LINEAR  WHICH  NONLINEAR 
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600  * 


T 


700  4 - — 4 

10  20 
p 

y  ■  «.+  01*1  (T)  +  a^P)  +  e  =£>  BALANCED 


14,000 
<2.000 
PT 

7.000 
6.000 

30  36 

T/P 

y  ■  0o+  0|(PT)  +  0|<T/P)  +  e  =£>  UNBALANCED 


Note  that  we  are  considering  the  model  form 
BEFORE  designing  the  experiment  The  example 
here  illustrates  the  importance  of  this  step.  Imagine 
an  experiment  to  estimate  a  reaction  rate  that  depends 
on  temperature  and  pressure.  Let  y  represent  the  rate 
and  let  P  and  T  represent,  pressure  and  temperature, 
respectively.  Here  a  particular  design  is  a  full 
factorial  for  a  model  of  the  first  sort.  This  means 
that  each  value  T  run  in  the  experiment  is  run  in 
combination  with  each  value  of  P  run  in  the 
experiment. 

If  our  model  were  based  on  the  same  two  variables 
but  in  a  different  way,  say  PT  and  T/P,  then  the 
same  design  takes  on  a  decidedly  poorer 
appearance,  and  is  no  longer  a  full  factorial! 


Source!  Satterthwaite  (1959). 


DESIGN  GOODNESS  DEPENDS 
ON  THE  MODEL 


In  our  modeling  work  we  will  often  fit  polynomial 
response  models  to  simulation  model  outputs  as  a 
function  of  simulation  model  inputs.  These  will  take 
the  general  form: 


yj  *  ^  Pj  Xy+  6i,  6}  are  i.i.d.-  N(0,  o2) 

Usually  x0  ■  1,  and  the  x's  may  be  different 
independent  variables,  powers  of  independent 
variables,  or  products  of  independent  variables.  For 
our  temperature/pressure  example,  we  may  have 
Xu  -  T,  xi2  -  P,  xu  ■  T2,  xM  ■  P2,  xu  ■  TP,  and 
so  forth.  The  &  are  random  variations,  which  are 
usually  assumed  to  be  independent,  indentically 
distributed  Gaussian  random  variables.  We  will 
return  to  a  more  detailed  discussion  of  this  'General 
Linear  Model'  in  Section  V,  where  we  also  mention 
some  alternatives  for  metamodels  of  simulation 
model  outputs. 
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DEFINITIONS 


iJeflnilion:  An  experiment  is a set  of  one  or  more  runs  (of  a  simulation 
model)  made  lo  meet  n  particular  set  of  objectives. 

'definition:  An  Independent  variable  is  a  parameter  of  die  (simulation) 
system  tliut  can  be  explicitly  adjusted  by  the  experimenter. 

(definition:  A  design  factor  is  an  independent  vuriablc  that  will  have  its 
value  changed  during  (lie  course  of  an  experiment. 

Definition:  A  design  frutnc  is  a  specification  of 

•which  independent  variables  will  be  held  fixcd(&  value) 
•design  factore(&  ranges) 

•what  system  outputs  will  be  measured, 


Definition:  An  experiment  design  is  a  set  of  specifications  of  design 
factors  for  an  experiment,  along  with  a  single  specification 
(vector)  for  the  Independent  variables  that  are  not  design 
factors. 

Definition:  A  design  ntalrlxdepends  on  the  model  to  be  fitted  at  well  as 
on  the  experimental  conditions,  There  Is  a  column  in  the 
matrix  for  each  term  in  the  model  to  be  fitted,  and  a  row  for 
each  (simulation  model)  run.  Each  row  of  the  matrix  holds  ilte 
values  of  the  model  temti  for  the  corresponding  ran. 


III.  MULTIDIMENSIONAL  POINT  PLOTS 

A.  Review  of  Design  Terminology 

Our  main  focus  in  this  section  will  be  on  graphical 
methods  for  developing  designs.  First  we  will 
review  some  basic  concepts  and  terminology  from 
the  field  of  experiment  design.  We  will  use  many  of 
these  terms  in  our  discussions  later  with  no  further 
explanation. 


TERMS  AND  TOPICS  IN  EXPERIMENTAL  DESIGN 


Nuisance  varUMae: 

BxampUi;  random  *  aetd,  auittng  conditio ni,  dina 

Blocking,  nature  of  Influence  undaratood 

(treat  Ilka  any  othar  design  factor) 

Randamlaitlon.  nature  of  Influaitoa  not  undantood 


Factorial  Daalgni: 

An  experiment  tun  for  each  poutbte  combination  of 
factor  lavalr.  If  all  facton  at  two  itvels,  than  runs 
eoireapand  to  vertical  of  an  n-d  intention  a)  cube. 

Fractional  Factorial  Dasipu-  not  all  veidc**: 


Pull  Factorial  Fractional  Factorial 

2>  (3  factor*)  (I  facton) 
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EXAMPLE: 


Model:  y=tto  +  a,x,  +a2x2  +  a3x,x2  This  illustration  of  the  design  matrix  also  gives  us 

our  first  graphical  design.  It  is  a  'factorial'  design. 
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Another  important  class  of  designs  are  'mixture* 
designs.  In  chemical  formulation  problems,  one 
often  has  to  have  the  components  add  to  100%. 

Thus  arbitrary  combinations  are  not  possible.  This 
kind  of  experimentation  can  be  important  for  strategic 
models  as  well,  where  the  total  resource  pool  is 
fixed,  and  the  choice  is  how  to  deploy  the  resources 
or  what  to  purchase  with  them. 

Source:  Scheffd  (1958) 
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TERMS  AND  TOPICS  IN  EXPERIMENTAL.  1)1, SION 


Confounding: 

Dciign  won't  allow  independent  eitinteiion  of  two 
nr  more  model  parameter! 

Example:  change  two  variable!  from  previoua  run, 
and  limulatlon  output  improve!.  Which  change  (or 
both)  caused  the  Improvement? 

Resolution! 

A  mithemiilcil  measure  of  the  nature  of  confounding 
lor  vnrloui  fractional  factorial  doilgm, 

Ifisi: 

What  If  postulated  model  Is  wrong?  Whst  impact  on 
parameter  eatimatei? 

Optimal  Design: 

Optimal  in  a  limited  mathematical  tense,  Definitions 
of  optimality  based  on  propeniei  of  the  matrix  XTX, 

(  X  It  the  design  matrix) 

Example:  D-Opilmality  <■■>  msx  determinant  ( xTX) 
x 

(for  a  fixed  number  of  rows  In  x) 
Optimality  depends  on  the  particular  model, 


For  practical  design  problems,  'optimal*  designs 
often  do  not  make  sense,  because  there  may  be 
considerations  or  constraints  that  cannot  be  described 
mathematically. 


These  figures  remind  you  of  the  issues  of  bias  and 
confounding.  For  the  first  pair,  the  design  at  the 
right  guards  against  bias  in  the  first  order  terms 
caused  by  a  nonzero  interaction  (xixj)  term.  The 
second  pair  illustrate  full  and  partial  confounding  of 
two  effects. 


COMPLETE 

CONFOUNDING 


PARTIAL 

CONFOUNDING 
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"Definition  13.1 :  A  txn  matrix  A  with  entries 
from  a  set  S  of  s  symbols  Is  called  an 
orthogonal  array  of  size  n,  t  constraints,  s 
levels,  strength  d,  and  Index  X  If  any  dxn 
submatrix  of  A  contains  all  sa  possible  dxl 
column  vectors  based  on  s  symbols  of  S 
with  the  same  frequency  X." 


-Raktoe,  Heydayat,  and  Federer 


Recall  that  your  main  strategies  for  choosing  a  design 
are  cookbook,  software,  and  graphical.  Bach 
approach  has  advantages,  ana  none  should  be  used 
exclusive  of  the  others.  One  advantage  of  graphical 
design  is  the  clarity  of  the  presentation.  This  makes 
it  easy  to  communicate  your  study  to  others  in  your 
organization  or  to  your  clients,  Compare  the  clarity 
of  idea  with  the  mathematical  description  at  the  left 


Historically,  graphical  methods  for  experiment 
design  have  not  been  recognized  as  an  entity.  A 
computerized  literature  search  of  scientific  journals 
gave  zero  entries  with  keywords  graphical  and 
experiment  design  in  the  last  ten  years.  Yet 

Shical  methods  have  been  used  by  outstanding 
sticians  to  develop  well  known  designs, 
including  Box's  central  composite  design.  Let's 
look  at  some  of  these  designs  and  see  what  we  can 
learn  about  creating  our  own. 
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B.  Examples  of  Graphical  Design 

Lets  begin  with  the  simplest  and  most  frequently 
used  class  of  designs:  factorial  designs.  These  are 
applicable  when  we  have  quantitative  variables  with 
simple  upper  and  lower  bounds  on  reasonable 
values. 


Here  we  return  to  the  Andrews  (1964)  SPAM  study. 
The  graphical  representation  here  is  very  simple, 
with  designs  for  experiments  involving  one,  two, 
three,  or  four  independent  variables  (factors).  Note 
that  the  four-dimensional  case  is  represented  as  a 
pair  of  cubes.  We  will  use  this  approach  to  generate 
even  higher  dimensional  designs. 
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A  2*  Factorial  Design 
(ft  4-tltfk  uytuti  Itftfii  If  Iffiftf.  If  WffUfl) 


Here  we  see  a  template  for  a  2$  factorial  design.  For 
experiments  with  many  factors,  we  will  usually 
choose  to  run  only  a  fraction  of  these  points.  This 
diagram,  then  can  be  thought  of  as  a  template  for 
identifying  conditions  that  will  be  run.  At  these 
vertices,  we  will  draw  a  circle,  square,  triangle,  or 
some  other  marker. 

How  do  we  decide  which  comers  to  choose?  Of 
course,  one  could  use  a  standard  fraction,  choose  a 
defining  relation,  or  use  a  computer  package  to  do 
this.  We  will  illustrate  here  how  to  do  it 
graphically. 
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Consider  these  two  designs  described  in  two  ways 
by  Youden  (1962, 1972).  The  design  representation 
at  the  top  makes  it  clear  that  the  one-at  a  time  strategy 
is  inferior  in  terms  of  covering  the  design  space  to 
the  two  variables  at  a  time  design.  The  preferred 
design  is  shown  in  tabular  form  In  the  lower  figure. 
Its  advantages  are  no  longer  apparent  It  is 
interesting  to  note  that  Youden  presented  the  clearer 
representation  ten  years  aA'r  the  tabular  form.  Do 
you  think  he  created  the  design  using  the  table  or  the 
graph?  Which  do  you  think  would  be  easier? 

What  advantages  do  you  see  in  the  two-factors  at  a 
time  design? 
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inMIIHO  vauiu  ' 


♦ 


CHANGE  TWO 
VARIABLES 


Consider  these  two  designs  for  estimating  the  effects 
of  three  independent  variables,  X,  Y,  ana  Z  on  a 
dependent  variable,  say  W. 

Source:  Youden(1972) 

Q:  Why  might  you  say  that  the  design  on  the  right  is 
'better'  than  the  design  on  the  left? 

A:  Lots  of  ways  to  answer.  Jot  down  yours: 


a.  I.  r.  10 K  AND  j.  I,  HUHIU 


l''iouiig  a— I'rolwtion  ot  3V||  Into  throe  3>  lictsrliii, 


This  figure  illustrates  that  each  projection  is  a  full 
factorial.  That  means  if  any  one  of  the  three  factors 
has  insignificant  effect  on  W,  we  will  have  gotten  a 
full  factorial  design  on  the  other  two  factors  for  free! 
Box  and  Meyer  (1986)  call  this  'exploiting  effect 
sparsity'.  That  is,  when  we  study  many  Factors,  we 
expect  that  most  of  them  will  have  little  effect  on  the 
dependent  variable  (here  called  W). 

Q:  How  do  we  check  that  we  will  be  able  to  exploit 
effect  sparsity? 

A:  Look  at  the  projections  of  the  design.  Do  they 
yield  full  factorials  or  at  least  good  fractions? 

Q:  What  does  a  'good  fractional  design'  look  like? 

A:  Check  the  confounding  patterns 

Source:  Box  and  Hunter  (1961) 
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These  diagrams  can  be  viewed  in  two  ways.  First, 
consider  an  experiment  with  two  independent 
variables,  X  and  Y.  We  will  study  their  effect  on  a 
dependent  variable  W.  To  compute  the  main  (linear) 
effect  of  X,  we  would  subtract  the  average  of  the  W 
values  measured  at  the  light  circles  from  the  average 
W  values  at  the  dark  circles  on  the  leftmost  figure. 
The  XY  interaction  effect  on  W  would  be  measured 
by  subtracting  the  average  W's  at  the  light  circles 
from  the  average  at  the  dark  circles  for  the  rightmost 
figure. 


MAINBFFHCT 


two-factor  (To  see  this,  label  the  low  and  high  X  and  Y  values 
interaction  as  q  and  1  respectively,  substitute  the  values  in  the 
model  a+bX+c  Y +dXY  to  get  a  formula  for  each  of 
the  four  points,  and  then  perform  the  subtraction) 


•  -  O  - 

MAIN  EFFECT 

AMD 

Z 


NOW  view  the  figure  in  a  different  light.  Suppose 
that  the  DARK  circles  correspond  to  a  HIGH/ value 
of  a  third  independent  factor,  Z.  How  do  we  * 
measure  the  main  effect  (linear  coefficient)  for  Z?  By 
subtracting  the  average  of  the  light  circles  from  the 
average  of  the  dark  circles.  For  the  design  at  the  left, 
the  same  auantity  is  also  used  to  estimate  the  main 
effect  for  X.  Thus  for  the  design  at  left,  we  will  not 
be  able  to  separate  the  effects  of  X  and  Z.  We  say 
that  the  main  effect  of  X  is  completely  confounded 
with  the  main  effect  of  Z. 

Q:  In  this  new  light,  which  of  these  designs  is  to  be 
preferred? 

A:  The  design  on  the  right 


•  -  O 


Q:  Why? 


two-factor  A:  Statisticians  assume  (sometimes  wrongly)  that 
tOTBSAcnoN  second  order  effects  (nonlinear)  will  be  less 

Important  than  first  order  (linear)  effects.  Thus  the 
z  second  design  confounds  the  coefficient  of  Z  with 

the  XY  interaction  coefficient,  and  the  XY  interaction 
coefficient  is  assumed  to  be  smaller  and  less 
important  than  any  of  the  main  effects  for  X,  Y,  or 
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We  have  learned  several  graphical  lessons  from  these 
simple  designs: 

•  check  projections  for  goodness 

•  look  for  patterns  of  confounding 

•  make  the  design  points  'cover'  the  design  space 

•  make  the  design  points  as  far  from  each  other  as 
possible 

Let's  try  to  apply  this  information  to  the  'Bad  Block’ 
example  from  Box,  Hunter,  and  Hunter  (1978). 

Consider  an  experiment  to  estimate  this  effect  of  three 
independent  factors,  a,  b,  and  c,  on  a  dependent 
variable  e.  Furthermore,  there  is  a  nuisance  variable, 
d,  that  forces  us  to  block  our  experiment  to  try  and 
provide  equal  values  of  d  within  a  block. 
Unfortunately,  the  homogeneous  block  capacity  is 
only  two,  so  we  can’t  run  all  8  combinations  of  the 
other  three  factors  in  one  block.  How  should  we 
assign  runs  to  blocks? 

The  figure  at  the  top  of  the  page  shows  two  designs 
represented  in  the  typical  tabular  fashion.  The  three 
columns  on  the  left  correspond  to  one  design,  the 
three  columns  on  the  right  to  another.  For  example, 
the  first  design  has  a  high,  b  high,  and  c  low  for  the 
first  point  in  block  one,  and  a  and  c  high  and  b  low 
in  the  second.  Can  you  tell  which  of  these  two 
incomplete  block  designs  is  defective?  Can  you  tell 
why? 


The  figure  on  the  lower  half  of  the  page  shows 
graphical  representations  of  the  same  two  designs, 
along  with  the  three  projections  found  by  dropping 
either  a,  b,  or  c. 


JD 

US  m  U0: 00: 

i  •  •  1 

BLOCK  1  BLOCK,  BLOCK  1  ULC0K4 


BLOCKED  DE8IQNS  FROM 
BOX,  HUNTER  &  HUNTER, 
(pp339-341 ) 


Q:  What  are  the  blocks  confounded  with  in  the  top 
design?  In  the  bottom? 


Q:  As  a  statistician,  which  design  would  you  prefer, 
making  the  normal  assumptions  about  mam  effect  vs 
second  order  confounding? 

Notice  also  that  the  top  design  violates  our  other 
design  principles:  the  points  on  a  cube  are  not  as  far 
apart  as  possible,  nor  do  they  cover  the  space  as  well 
as  possible. 
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OR 
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a: 
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Confounding  of  effort  £  with  main  affaotj  If 

■  muni  vdtl 
•  mum  olo 


□*  □* 
a  • 

dlo  d  hi 


Confounding  of  parity  of  «-i  confounding 
with 

main  affaet  d  In  a  d-factor  doiign 


The  graphical  patterns  of  confounding  ate  based  on 
the  graphical  patterns  for  computing  main  effects  and 
higher  order  interactions.  There  are  two  graphical 
operations  that  induce  confounding.  The  first,  which 
we'll  call  confounding  of  affect,  superimposes  the 
graphical  pattern  for  one  effect  in  the  same  fashion  as 
another.  In  the  first  example  at  the  left,  the  hi-lo 
pattern  for  main  effect  c  is  the  same  as  the  hi-lo 
pattern  for  main  effect  a  or  an  ac  confounding. 


The  second  figure  illustrates  another  wav 
confounding  patterns  may  appear  graphically,  which 
we  call  confounding  of  parity.  Here  we  have  four 
factors,  with  factor  c  coded  by  squares  and  circles. 
The  c  main  effect  is  confounded  with  a  again  within 
a  single  level  of  d,  but  the  sense  of  the  confounding 
(parity)  reverses  between  the  lo  and  hi  values  for  d. 
This  gives  an  acd  confounding  term  in  the  defining 
relation  for  the  design. 


3?8 
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We  can  evaluate  even  larger  designs  graphically,  but 
first  we  need  to  look  at  higher  order  confounding 
patterns.  The  figure  at  the  left  is  from  Box,  Hunter, 
and  Hunter  (1978)  and  shows  the  interaction  patterns 
for  3-factor  experiments.  Thus  if  we  color  code  a 
fourth  factor  on  a  three-factor  plot,  the  color  pattern 
will  convey  which  effect  the  fourth  factor  is 
confounded  with.  In  fact,  we  can  carry  this  tool  to 
higher  dimensions  by  looking  at  some  (not  all)  of  the 
confounding  patterns. 


d  a 


d 

confounded 
with 
a  • 


d 

confounded 

with 

ac 


d 

confounded 

with 

abc 


I.ONI'OtlNItlNti  I'ATTKKNNi 


oirrau 

iNNim 

tell#: 

*  m 

idfg: 

{ 


r  Here  is  a  fractional  factorial  design  from  Fries  and 
Hunter  (1980).  Can  you  pick  out  interaction  patterns 
graphically? 


4> 


The  be  interaction  pattern  on  the  small  cubes  has 
been  circled.  This  pattern  falls  on  the  dg  interaction 
pattern  on  the  large  cube  (the  alternate  dg  vertices 
have  the  other  be  interaction  pair).  Thus  be  is 
confounded  with  dg,  as  shown  in  the  second  term  of 
the  defining  relation.  Similarly  on  these  cubes,  the 
main  effect  of  a  is  confounded  with  the  main  effect  of 
f.  The  opposite  pattern  occurs  on  the  complementary 
dg  interaction  nodes.  Thus  dg  is  confounded  with 
af,  the  third  term  In  the  defining  relation.  Finally,  the 
a-f  main  effect  confounding  switches  sense  for 
opposite  sets  of  the  be  interaction  (we  already  knew 
this  since  be  is  confounded  with  dg,  but  we  can  see  it 
graphically  directly).  Thus  af  is  confounded  with  be, 
the  first  term  in  the  defining  relation. 


d 
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Q:  What  should  be  our  goal  in  developing  a  better 
design  graphically? 

A:  Lay  out  our  points  in  patterns  that  cotrespond  to 
high  order  interactions,  not  main  effect  patterns. 

In  this  figure,  we  are  able  to  generate  a  pattern  that 
corresponds  to  three-way  interaction  on  the  large 
cube  vertices,  but  the  small  cube  vertices  still  snow  a 
main  effect  (a)  pattern  which  is  confounded  with  the 
large  cube  three  way  interaction  (deg).  Thus  we 
have  an  adeg  term  in  the  defining  relation. 


This  third  attempt  is  deceiving!  It  appears  that  we 
have  done  it,  getting  three-way  interaction  patterns 
on  both  the  small  and  large  cube  vertices.  But  we 
have  forgotten  the  f  main  effect  (square  vs  circle).  It 
is  completely  confounded  with  the  three  factor 
interacdon  (deg)  on  the  large  cube  vertices.  Thus  we 
still  have  one  four-letter  word  for  this  fractional 
design!  At  least  it's  repeatable:  defg. 


Note:  these  designs  appeared  in  the  original  article, 
but  the  graphical  representations  did  not.  The  last 
design  was  cited  by  the  authors  as  the  'minimum 
aberration'  design  because  it  minimized  he  number 
of  words  in  the  defining  relation  that  had  minimum 
length. 
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Exercise  5:  Using  the  template  below,  fill  in  points  for  a  'good'  1/2  fraction  design. 
(Hint:  check  projections  and  interaction  patterns  as  you  go). 
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a 
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Z 
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1 

7 

a 
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Exercise  6:  Can  you  'fix'  this  design  by  moving  just  one  point? 
ABODE 

+  -  + 

-  +  + 

+ 

+  ~  + 

-  +  -  -  + 

+  +  +  -  + 

+  -  —  +  “ 

-  -  +"+  - 

+  +  +  +  - 

+  -  -  +  + 

+  -  +  +  + 

—  4-  +  +  + 

■|«  (N  *|l 


Exercise  7:  You  have  a  model  for  a  response  W  as  a  function  of  two  parameters,  X,  and 
Y.  You  expect  the  response  to  be  nonlinear,  and  so  a  two-level  factorialls 
not  satisfactory.  You  nave  enough  resources  to  run  9  experiments,  so  a  32 
factorial  design  is  possible.  Unfortunately,  your  available  design  space  is 
not  rectangular  (see  below).  Lay  out  your  9  runs  within  the  design  space 
provided  below: 


INDEPENDENT 
VARIABLE  X 

Box  represents  allowable  range  for  values  of  X  and  Y 
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Q:  How  do  we  choose  the  design  points  in 

.  .  .  complicated  situations  when  the  factors  can  take  on 

..  convanlant  to  ragard  daiigni  at  built  up  from  a  numbar  three  or  more  levels?? 

of  component  aatt  of  points,  aach  a  at  having  Ita  points 

tquldlstanl  from  tha  origin ..."  A:  - -j  , 

"...  form  tha  vartlcas  of  a  ragular  polygon,  polyhadron,  or  1 

polytops..." 

•Box  and  Huntar  (1957)  ' 


This  figure  shows  a  graphical  design  from  the  first 
paper  in  the  first  issue  of  Technometrics  (DeBaun, 
1959).  This  design  is  broken  down  into  graphical 
subcomponents  to  make  it  easier  to  understand. 


O  OCTAHEDRON 
•  CUBOCTAHEORON 
®  CENTERPOINT 


Fiouna  1 
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Hunter  (1985)  illustrates  many  common  designs 
graphically  as  the  vertices  or  edge  points  of  regular 
polyhedra. 
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Let's  summarize  our  discoveries  about  how  to 
generate  good  designs  graphically.  These  ideas  are 
not  new,  as  these  quotes  illustrate. 


Remember,  it  is  easier  to  view  a  complicated  design 
as  being  made  up  of  simpler  graphical  components. 

\.  convenient  to  regard  designs  as  built  up  from  a  number 

of  component  sets  of  points,  each  set  having  its  points  Source.  BOX  and  Hunter  (1957) 

equidistant  from  the  origin  ...” 

”...  form  the  vertices  of  a  regular  polygon,  polyhedron,  or 
polytope...” 
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"Chooit  n«w  points  to  MAXIMIZE  th»  minimum 
dlstanc#  from  all  axlsting  dtilgn  points..* 


Also,  keep  design  points  as  far  apart  as  possible: 
span  the  design  space. 


Source:  Kennard  and  Stone  (1969) 


"...It  la  provad  (Appendix  t)  that  II  a 
polynomial  ol  any  dagraa  d,  is  fitted  by  tha 
mathod  ol  laaat  aquaraa  over  any  region  ol 
Inlaraat  fl  In  lha  k  variable*,  whan  the  true 
lunetlon  la  a  polynomial  ol  any  degree  dt  »  dv 

than  lha  blaa  averaged  ovar  fl  la  mlnlmixed  lor 
all  values  ol  the  ooatflolanta  ol  tha  neglected 
terma,  by  making  the  moments  ol  ardor  dt+dt 
bnd  lesa  ol  tha  daalgn  points  equal  to  the 
corresponding  momenta  ol  a  uniform 
distribution  over  fl," 


Unfortunately,  the  Box  and  Draper  goal  is  in  conflict 
with  measures  taken  to  minimtae  bias.  To  minimize 
bias,  the  distribution  of  points  in  space  should  appear 
to  come  from  a  uniform  distribution.  A  uniform 
distribution  does  not  put  points  at  the  extreme 
locations,  but  spreads  them  evenly  over  the  design 
space.  Thus  there  will  be  a  tension  between  the 
second  and  third  goals  when  you  design 
experiments. 


Source:  Box  and  Draper  (1959) 


SOME  USEFUL  CONCEPTS 
lor  generating 
GOOD  DE9IQNS 
Irom 

MULTIDIMENSIONAL  POINT  PLOTS 


This  table  presents  our  findings  on  generating 
graphical  designs.  Let's  apply  these  tools  to  a  real 
design  problem. 


•  COVER  THE  DESIGN  SPACE  UNIFORMLY 

•  CHECK  PROJECTIONS  TO  PLAMBB  * 

AND  LINES 

'  SPAN  THE  WHOLE  DBHIOM  SPACE: 

MAKE  ADDED  DESIGN  POINT8  PAR  FROM 
EXISTING  POINTS  TO  MINIMIZE  VARIANCE 
FOR  FIRST  ORDER  EFFECTS 

•  DECOMPOSE  COMPLICATED  DESIGNS  INTO 
GRAPHICAL  SUBCOMPONENTS 
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We  return  to  our  discussion  of  the  disk  pressing 
problem.  Suppose  we  have  narrowed  the  list  of 
factors  to  vary  to  the  list  shown  at  the  left.  Upper 
and  lower  limits  are  specified  for  each  independent 
variable,  and  whether  they  appear  to  have  linear  or 
quadratic  effect  on  disk  warp  values. 


REOUCEO  LIST  OF  INOEPENOENT  VARIABLES 
FOR  THE 

OISK  PRESSING  STUDY 


VARIABLE 

LOWER  UM 

UPPER  UM 

NONTIN? 

GLASS  SWITCH 

4.900- 

4  925- 

UNEAR 

RAM  VELOCITY 

LOW(-l) 

HIGH(*1) 

LINEAR 

COOLING  TIME  (T-12) 

12  SEC. 

IS  SEC. 

LINEAR 

LOW-HIGH  TRANSITION 

200PSI 

EOOPSI 

NONUN 

FULL  CLAMP  PRESSURE 

isoopsi 

2000PSI 

NONUN 

Before  we  design  the  main  ’exploratory'  experiment, 
we'll  try  to  design  a  small  'pilot'  experiment  The 
purpose  of  the  pilot  experiment  is  to  verify  that  the 
proposed  ranges  for  the  independent  variables  are 
feasible  (we  don't  break  the  press)  and  to  debug  the 
experiment  running  and  data  collection  process.  It  is 
analogous  to  kicking  the  tires  and  checking  the 
brakes  before  beginning  a  long  trip. 


E.XPE.Ritt£tfTS 


VJliAT  AR£  THtf  ? 

WHAT  AK£  TH6V  FoR? 

HOW  IMPORTANT- 

A  «<£■  TB  £  V  ? 
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Exercise  8:  You  have  a  total  of  six  runs  available  for  a  pilot  experiment.  Identify  the 
six  runs  you  choose  to  make  graphically  ana  state  why  you  chose  them. 
Below  is  a  filled-out  full-factorial  design  requiring  72  runs,  followed  by  a 
template  for  you  to  use  to  place  your  design  points. 

(Spend  5  minutes  individually,  then  5  minutes  in  groups) 


FULL  FACTORIAL  DESIGN 


□  4,835 

GLASS  SWITCH  (QS): 

•  4,800 


OLAMP  PRESSURE  (OP) 


LOW  - —  RAM  VELOOITY  - —  HIGH 

DESIGN  TEMPLATE 


□  4,825 

GLASS  SWITCH  (GS): 

•  4.800 


CLAMP  PRESSURE  (CP) 

LOW  -  RAM  VELOOITY  -  HIGH 
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Exercise  9:  You  have  a  total  of  twenty-one  runs  available  for  an  exploratory 

experiment.  You  will  want  to  fit  a  model  that  is  linear  in  three  terms  (glass 
switch,  ram  velocity,  and  cooling  time)  and  quadratic  in  two  variables 
(low-high  transition  pressure  and  full  clamp  pressure).  Identify  the 
twenty-one  runs  you  choose  to  make  graphically  and  state  why  you  chose 
them. 

(Spend  10  minutes  individually,  then  10  minutes  in  groups) 


DESIGN  TEMPLATE 


□  4.925 

GLASS  SWITCH  (GS): 

•  4.900 


LOW  -  RAM  VELOCITY - HIGH 
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C.  Constructing  a  Design  for  the  Main  Example 

This  figure  shows  the  full  72-run  factorial  design  and  a  36-run  1/2  fraction.  How  was  the  1/2  fraction 
chosen?  The  breakdown  into  graphical  subcomponents  is  made  clear  on  the  following  page. 

FULL  FACTORIAL  DESIGN 


□  4.925 

GLASS  SWITCH  (GS): 

•  4.900 


FRACTIONAL  DESIGN 
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For  3-level  designs  such  as  this  one,  confounding  properties  are  much  harder  to  detect,  and  will  require 
the  mathematical  checks  discussed  in  Section  V.  The  figure  below  shows  a  modified  design  with 
improved  confounding  patterns.  This  was  generated  by  Master  of  Engineering  candidates  after  a  one  hour 
tutorial  on  these  graphical  techniques  (Young,  Moore,  and  Girard,  1987). 


nNAW,.BUN 


■«  . .  . fr 

IMS  1700  1IM 


■«  . .  . fr 

1100  1700  1IM 

CP 


P  •  NUNt  THAT  DUPUCATtD  PLOT  MJNO 

IP  .  HUN  WU  OUPUOATBO  M  PNAL  WPWIMBff  AN0  0UPUCATO  PHOf  PWN0 


Figure  12 
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Exercise  10:  Use  graphical  methods  to  design  an  experiment  to  fit  a  response 

metamodel  to  a'simulation  system  of  your  choice.  The  design  should  be 
for  at  least  three  independent  variables,  at  least  one  of  which  is  at  more 
than  two  levels.  At  least  ten  runs  should  be  planned. 
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D.  Displaying  Results  on.Graphical  Designs 

Graphical  designs  provide  another  advantage:  they 
provide  a  frame  on  which  to  visualize  the  system 
responses.  This  allows  one  to  interpret  the  findings 
more  easily  and  to  propose  appropriate  models  for 
the  response  function  more  easily. 

The  simplest  examples  of  such  a  display  are  shown 
here.  The  first  is  from  Snee  (1985a).  Pearson 
(1934)  draws  simple  examples  of  brick  strength  vs. 
kiln  location.  Diagrammatic  Section  or  Kiln 


naunaa  in  BViMNaTc  aM  avbnaw 
•r  immm*  in  ba.paH'M.aff.  wi  a 

i  i 

This  figure  is  from  Bates  (1989).  Elaborate  physical 
design  models  are  Illustrated  below  and  on  the 
following  page. 

Sources: 

below:  Box  and  Wilson  (1951) 
next  page:  Neyman(1935) 

Box's  physical  model  of  the  original  central 
composite  design  was  less  complicated  than  the 
1  figure  below.  A  simple  bare  wire  structure,  it  was 
augmented  to  display  response  values  using  insulated 
wire.  Each  vertex  was  wrapped  with  a  piece  of 
insulated  wire.  The  color  or the  insulation  was 
keyed  to  the  level  of  response,  from  red  to  blue  to 
white.  Design  points  with  intermediate  values  of  the 
response  function  were  wrapped  with  the  two 
adjacent  colors  (red&blue  or  olue& white). 

I  III 

l  (Model  on  loan  from  J.  Stuart  Hunter) 


FIGURE  1(1.  PHOTOGRAPH  OF  A  THREE  DIMENSIONAL  MODEL  SHOWING  THE  CON¬ 
TOURS  OF  THE  APPROXIMATE  PLANE  STATIONARY  RIDGE  SYSTEM  CONSTRUCTED 
US  THE  OASIS  OF  TRIALS  RUN  AT  THE  POINTS  INDICATED  JIY  THE  MARBLES.  A 
DESCRIPTION  OF  THIH  PARTICULAR  EXPERIMENT  IS  FOUND  IN  SECTION  8.  THE 
TV  PE  OF  EXPERIMENTAL  DESIGN  USED  IS  THAT  ^IjUSTIlATED  IN  FIOURE  8. 
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C:  conjugate  gradient  (algSOO) 
Algorithm  8:  BFGSOigSGO) 

6:  Gay's  6FGS  (»1g6l  l> 

0.  Davioon  optimally  concitionco 


Another  way  to  display  the  level  of  response  is  with 
the  size  of  the  ball  at  the  vertex.  For  two-factor 
experiments  at  two  or  more  levels,  this  display  is 
called  a  bubble  chart.  This  example  is  for  an 
experiment  comparing  the  performance  of  several 
types  of  optimization  algorithms  with  several 
strategies  for  computing  finite  difference  derivatives 
(Barton,  1990).  A  larger  circle  means  that  the 
effectiveness  was  higher. 


Figure  9.  Diameter  of  circlet!  represents  average  rank  of  function  reduction  for  each 
finite  difference  method  on  18  test  functions,  with  test  function  arguments 
shifted  by  100.  Rank  comparisons  over  all  optimization  codes.  Function 
accuracy  2  decimal  digits,  shifted  starting  points. 


FIRST  30  ITERATIONS 


This  figure  from  Barton  (1985)  shows  the  contours 
of  the  fitted  response  function  superimposed  with  the 
sequential  simplex  (Nelder-Mead)  design  points. 


It  is  also  possible  to  label  the  design  points  with  their 
run  order,  to  look  for  confounding  problems.  The 
figure  at  the  left  is  from  Snee  (1985a).  A  better 
approach  would  be  to  shade  the  circles 
corresponding  to  run  order,  or  to  size  them  from 
small  circles  to  large  based  on  run  order.  This  would 
help  to  make  confounding  patterns  more  obvious. 
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IV.  EXERCISE  REVIEW; 
A.  Fishbone/IDEF  Diagrams 


B.  Multidimensional  Point  Plot 
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VAUDATINQ  QEfllfltjS  THAT. HAVE  BEFN  fiREATER  natpuirn  i  v 
GRAPHICAL  TOOLS  EXPLOIT  THE  POWER  Of  THE  "RIQHT  BRAIN* 

:»r 

BUT  THEBE  TECHNIQUES  ARE  QUALITATIVE  AND  PRONE  TO  ERROR 
WE  NEED  MATHEMATICAL  CHEOKSi 


A  (j£M£KAL  L/AfEAR  MODEL 


y-  Ao  A*  «!,*,  ■*«  A,  Kt  4  AjXj  +  ♦  «rv»r 


Glv'ror  * 


dcs'6n  hat*ik 

X 
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V.  VALIDATION  OF  EXPERIMENT  DESIGN 

These  techniques  allow  us  to  use  the  creative  human 
graphical  processing  resources  to  create  new  designs 
and  to  gain  insight  on  design  properties.  Graphical 
methods  are  necessarily  qualitative,  however.  We 
need  to  back  up  bur  graphical  work  with  precise 
mathematical  measures  of  design  goodness. 


A.  Importance  of  the  Model 

As  we  discussed  earlier,  the  properties  of  an 
experiment  design  depend  closely  oh  the  kind  of 
model, tile  experimenter  has  chdsen.  What  properties 
do  we  care  about?  These  include 

•  estimability  of  parameters 

•  the  variance  of  the  parameter  estimates 

•  the  covariance/correlation  of  " 

•  the  mean  square  prediction  error  over  the  design 
space 

The  most  common  applications  in  simulation 
metamodeling  will  be  General  Linear  Models: 
ANOVA,  ANOCOVA,  or  Regression  models  with 
ild  Gaussian  error.  There  are  other  interesting 
choices  for  metamodels,  particularly  for  deterministic 
computer  simulation  output  (Sacks,  Schiller,  and 
Welch,  19S9).  At  present  these  models  require 
computer-aided  selection  of  design  points. 


B.  The  General  Linear  Model  and  the  Design  Matrix 

We  will  often  represent  a  single  dependent  variable 
(say  WARP)  as  a  y,  and  the  independent  variables  as 
x's.  In  this  formulation  the  unknown  parameters  are 
the  a's  and  the  value  of  c2.  Note  that  the  a's  appear 
LINEARLY  in  the  formulation.  The  x's  do  not  have 
to  be  linear,  as  illustrated  by  the  fourth  factor.  This 
could  also  be  written  as  X4  rather  than  (xt)2. 

The  observations  are  indexed  by  a  letter  (i),  and  the 
independent  variable  coefficients  are  indexed  by 
another  letter  (j).  The  x's  are  indexed  by  both  letters. 
The  representation  is  often  xjj,  rather  than  as  shown 
at  left  This  model  can  be  written  in  matrix  form. 
The  matrix  X  is  called  the  design  matrix. 
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Exercise  11:  Construct  the  design  matrix  (don't  forget  the  column  of  l's  for  the 
intercept)  for  the  following  design 


X2 

•  1  - 

i 

• 

X 

•  | 

i  ,  ■ 

-2  -1 

1  2 

•  -1  - 

• 

a)  when  the  model  is 

y  =s  aQ+a^+a-jX-j+error 

b)  when  the  model  is 

y  =  a0+a,xI+a2x2+a3x1x2+error 
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OPTIMAL  DESIGN: 

D-Optimallty 

maximize  ch^X-^) 

corresponds  to  minimum  volume  confidence  ellipsoid  for 
model  parameters,  i.e.  tightest  estimates 

E*Optimality 

maximize  minimum  eigenvalue  of  XTX 

corresponds  to  minimizing  maximum  prediction  error  over 
a  unit  sphere 


C.  Measures  of  Design  Goodness 

Measures  for  the  goodness  of  a  design  will  be 
difficult  to  generate  for  arbitrary  models  of  system 
response.  For  the  general  linear  model,  though,  we 
have  a  simple  situation:  the  goodness  ofthe  design 
for  fitting  a  general  linear  model  will  depend  only  on 
the  properties  of  the  X  matrix.  In  particular,  the 
estimate  for  the  variance-covariance  matrix  for  the 
parameter  estimates  (except  the  estimate  for  a?*) 
depends  on  (XTX)-1.  Since  the  x's  for  a  candidate 
design  are  known  before  the  results  are  collected 
(only  the  y's  are  unknown)  we  can  assess  the  quality 
of  the  design  before  the  experiment  is  run! 

The  list  at  the  left  presents  two  of  the  traditional 
definitions  cf  good  or  'optimal*  designs.  In  general 
we  would  like  the  diagonal  entries  or  Z  to  be  small. 
This  corresponds  to  small  variances  for  our 
parameter  estimates.  We  would  like  the  off-diagonal 
elements  to  be  even  smaller.  This  corresponds  to 
low  confounding  of  effects,  that  is,  small  covariance 
of  parameter  estimates. 


/  5AS  Code  to  £- 

CMS  FUlD  Lf  DtfSl  .DISK  D6SI6N  DAT  A  A; 

PA-TA  DESIGN) 

IWFILE  D6S1; 

/n p^r  At  u  as; 

XI XI  -  XirfXt,; 

XI  XL  '  XI*  XL, 

&rc 

PRJC  MATRIX; 

FfcTCH  A  DATA  «  DESIGN  ; 

Sic. fac  -  ihv  ((a')*  X), 

PRINT  x  $16  FAC; 


D.  Checking  (XTX)  and  (XTX)4 

The  code  at  the  left  is  for  SAS.  We  enter  the  design 
and  can  augment  it  if  we  have  nonlinear  x  terms  in 
the  general  linear  model.  Don't  forget  to  include  a 
column  of  l's  if  you  have  an  intercept  term,  We  can 
study  the  resulting  matrix  which  is  (except  for  the 
factor  o2 )  the  variance  covariance  matrix  L 


Other  software  such  as  MATLAB  ACED,  COED, 
etc.  can  be  used  to  generate  mathematical  properties 
of  design  goodness  for  general  linear  models. 
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Exercise  12:  Construct  the  design  matrix  (don't  forget  the  column  of  l's  for  the 

intercept)  for  your  own  problem.  Use  S  AS,  MATLAB,  ACED,  or  any 
other  software  to  evaluate  (X^X)"*. 
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E.  Graphical  Views  of  (XTX)  and  (XTX)-» 

While  it  is  easy  to  generate  these  matrices,  it  is 
difficult  to  understand  the  design  properties  for 
designs  with  more  than  two  or  three  parameters. 

Instead  we’ll  look  for  way  to  represent  these  matrices 
graphically!  Understand  that  these  graphics  are  for 
analysis  rather  than  generation  of  new  experiment 
designs.  They  are  used  for  checking  rather  than 
creating. 

We'll  begin  by  introducing  a  new  example.  We'll  try 
to  design  an  experiment  where  the  factors  are 
quantitative  but  integer,  and  the  realisdc  values 
correspond  to  small  integers.  An  analogy  might  be  a 
battle  deployment  plan  where  the  number  of 
divisions  to  assign  to  an  area  must  be  selected. 

In  our  example  the  simulation  model  represents  the  performance  of  a  small  manufacturing  operation.jrhe 
goal  of  our  study  is  actually  optimization,  but  it  was  pursued  by  Kleijnen  and  Standridge  (1987)  through 
fitting  a  polynomial  approximation  for  sensitivity  analysis  and  insight  (goal  2),  ^ '' 

The  figure  at  the  top  of  this  page  illustrates  the  manufacturing  system  layout  problem.  The  faefcry 
produces  a  single  product,  which  requires  three  tasks  to  be  performed  in  sequence.  Speclal  machines  are 
available  to  perform  each  of  these  tasks.  In  addition,  flexible  manufacturing  machines  fre  available  (at 
higher  cost)  which  can  peifoim  all  three  machining  operations.  To  meet  throughput  requirements  on  task 
1, 5.2  machines  are  required.  For  task  2, 1.3  machines  are  needed.  For  task  3, 2.6  machines  are  needed. 
We  cannot  purchase  a  fractional  machine,  and  so  without  any  flexible  equipment  we  will  need  six 
machines  for  task  1,  two  machines  for  task  2,  and  three  machines  for  task  3.  Oh  the  other  hand,  the 
purchase  of  one  flexible  machine  could  reduce  the  demand  on  one  or  more  other  tasks,  and  so  reduce  the 
number  of  machines  needed.  If  only  five  machines  for  task  1 ,  one  for  task  2,  and  two  for  task  3  arc 
purchased,  two  flexible  machines  will  be  needed  to  meet  throughput  requirements. 

The  goal  of  this  design  is  to  estimate  the  throughput  as  a  function  of  the  number  of  machines  of  each  type 
that  are  purchased.  This  is  expected  to  give  some  insight  on  the  mbst  economical  choice.  Of  course,  there 
are  only  2x2x2x3»24  possible  configurations,  so  the  exact  answer  can  be  had  for  the  cost  of  24  simulation 
runs.  We  assume  that  we  have  a  budget  of  only  8  runs  from  which  we  must  make  a  decision. 


MANUFACTURING  SYSTEM  LAYOUT 
DESIGN  PROBLEM 


TASK  I  TASK  2  TASK  3  TASK  1,2  OR  3 

MACHINES  MACHINES  MACHINES  MACHINES 


ORIGINAL  DBSION 


The  original  design  had  a  serious  problem.  Can  you 
see  it  graphically?  The  effect  of  the  number  of 
flexible  machines  (x4)  is  confounded  with  the 
number  of  machines  of  type  3  (x3~  here  re-coded  as 
±1  rather  than  2  or  3).  Since  the  confounding  is  only 

Jiartial,  it  is  hard  to  eyeball  how  severe  the  problem 
s.  We  need  to  look  at  the  covariance  and  correlation 
matrices. 
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liXl'IHUMI'.NTOISKiNS 
I ''OR  A 

MANtll'ACTOKINO  DOSION  t'ROIII.HM 


OUKJINAL  OG1IC1N 

I  XI  M  X)  X4  XU  X«  X44 


Matlab  Code: 


i  -i  1  -i 
1IN 
U-l-l 
Mil 
I  -1  -11 
11-11 
1111] 


B-  AJ(!.1),“AJ(:J)1 
C-  ( A3(:,l).*  A3<:,3)] 
D-  t  A3(;,l) ,*  AJ(;,4)] 
B-  [  A3(:,2)  .*  A3(:J)| 
F-  ( A3(:,2)  .•  A  j(:,4)] 
O-  [  A3(:,3)  ,*  A3(:,4)] 
H-  t  A3(:,4)  ,*  A3(:,4)| 


H-H-l 

AA3  ■[A3BCDBF0H] 

XTX-AA3’*AA3 

oorr(AA3) 


This  figure  shows  several  designs  along  with  the 
original  design.  Which  is  better?  In  what  way? 
These  are  difficult  questions  to  answer,  even  when 
the  variance-covariance  matrix  is  available.  For  only 
8  runs,  the  matrix  with  all  main  effects,  two  factor 
interactions,  and  a  quadratic  term  for  the  number  of 
flexible  machines  IS  NOT  INVERTIBLE.  To  study 
the  covariance  matrix  the  model  must  be  limited  to  a 
few  terms. 


The  second  figure  illustrates  the  entries  of  the 
variance-covariance  factor  (XTX)-1  for  the  original 
design.  The  edge  length  of  each  diagonal  blook  is 
proportional  to  the  square  root  of  the  diagonal  entry 
in  (X'X)-1.  Thus  the  size  of  the  diagonal  blocks  are 
proportional  to  standard  deviations  of  coefficient 
estimates  and  the  areas  are  proportional  to  their 
variances.  This  structure  determines  the  size  of  the 
off-diagonal  blocks.  Each  block's  shading  <s 
proportional  to  the  absolute  value  of  the  correlation 
coefficient  of  the  two  parameters  corresponding  to 
that  row  and  that  column.  The  on-diagonal  blocks 
are  shaded  100%  (black)  because  a  term's  correlation 
with  itself  is  1. 


Q:  How  was  this  graphic  created? 

A:  Using  Canvas™  for  the  Macintosh.  The 
software  allows  one  to  see  the  edge  dimension  of  a 
square  as  it  is  created.  It  also  provides  a  numerical 
scale  (0-100)  for  shading  boxes.  Matlab™  was  used 
to  generate  the  covariance/correlation  data.  The  code 
for  one  of  these  is  shown  at  the  left. 


Q:  What  will  a  good  design  look  like? 

A:  The  overall  size  of  the  box  is  determined  by  the 
sum  of  the  standard  deviations  of  all  estimated 
parameters.  The  smaller  the  better.  If  we  are 
particularly  interested  In  a  subset  of  the  parameters, 
this  sub-box  should  be  small.  Furthermore,  the  off- 
diagonal  shading  should  be  as  light  as  possible.  It 
should  be  white  In  the  sub-box  corresponding  to  our 
most  important  parameters  (no  confounding),  and  it 
should  be  pale  in  the  rows  and  columns  which 
overlap  the  sub-box  but  are  outside  it. 
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The  figures  on  this  page  show  the  graphical  representation  of  the  correlation/variance  structure  for  all 
candidate  designs.  The  important  effects  are  the  constant  (1),  the  quadratic  term  (X44)  and  the  linear  terms 
xj.  x3,  and  x4. 

Note  particularly: 

•  the  original  design  has  substantial  confounding 

•  the  overall  box  for  the  original  design  is  bigger  than  the  designs  at  the  right 

•  the  modified  original  design  eliminates  confounding  from  the  sub-box  of  interest  while  providing  the 
same  accuracy  on  parameter  estimates 

•  the  Kleijnen  box  is  smaller  overall  and  smaller  for  the  key  effects  that  it  does  estimate 

•  the  Kleijnen  design  cannot  estimate  one  of  the  key  parameters  (x^) 

«  the  expanded  X4  design  provides  both  a  smaller  box  and  estimabmty  of  (X44) 


EXPERIMENT  DESIGNS 
FORA 

MANUFACTURING  DESIGN  PROBLEM 


ORIGINAL  DESIGN  KLEUNEN  DBSION 

1  *1  XJ  *1  X4  XI4  XU  X44  I  XI  XJ  XI  X*  XI4  XM  XJ4 


MODIFIED  ORIGINAL  DBSION  EXPANDED  X4  DESIGN 
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This  graphical  tool  for  analysis  of  correlation  and 
covariance  needs  further  revision.  It  is  not  clear 
from  the  picture  that  the  Kleijnen  design  cannot 
estimate  the  quadratic  effect  This  is  important 
though;  it  is  like  throwing  the  baby  out  with  the 
bathwater, 

It  is  interesting  that  covariance  patterns  can  be  traced 
from  dark  cells  to  the  corresponding  variable  labels 
on  the  boundary.  If  the  matrix  could  somehow 
include  all  model  terms  (not  just  an  estimable  subset) 
one  could  construct  Taguchi  linear  graphs  by  reading 
the  dark  boxes  as  edges  between  variables.  An 
example  of  Taguchi linear  graphs  is  shown  here. 

This  is  also  a  good  reminder  that  the  Taguchi  Linear 
Graphs,  just  like  the  graphical  correlation/covariance 
matrices,  are  useful  for  analysis  (or  cookbook  look¬ 
up)  but  not  for  creating  new  designs. 


Source;  Pignatiello  and  Ramberg  (1985) 
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1  DEFINE  GOALS 

2  IDENTIFY  VARIABLES 


VI.  SUMMARY 

We  have  examined  graphical  methods  for  three  of  the 
five  steps  in  designing  experiments.  These  methods 
are  easy  to  learn,  fun  to  do,  and  provide  a  new 
dimension  of  insight  to  the  qualities  and 
consequences  of  a  particular  design. 


3  CLASSIFY  VARIABLES 


4  DESIGN  EXPERIMENT 

5  VALIDATE  DESIGN 


Graphical  designs  art  not  a  replacement  for  other 
design  techniques.  Like  other  techniques,  they  have 
their  own  advantages  and  disadvantages.,  But  until 
recently,  they  have  not  been  publicized. 

i  * 


STRENGTHS  AND  WEAKNESSES 
OP 

GRAPHICAL  METHODS 


+  FLEXIBLE 

•  MAKE  TRADEOFF*  VISUALLY 

*  INCORPORATE  CONSTRAINTS  GRAPHICALLY 


♦  ROBUST  (totdagrMjfor  Utonlar  modal! 


4  USES  POWERFUL  COMPUTING  DEVICE 

(human  vition  tyrttm  cad  rigt*  bode) 


+  EASY  TO  USE  AND  EASY  TO  REMEMBER 


+  EASY  TO  TEACH 


-  not  QUANTITATIVE 
*  DIMENSIONAL  LIMITATIONS 
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As  this  quote  from  George  Box  (1984)  suggests, 
graphical  methods  are  perhaps  more  important  for 
design  than  for  analysis,  since  design  is  a  highly 
creative  activity.  In  this  age  of  computerized 
statistical  packages,  errors  in  analysis  can  always  be 
redone,  but  an  error  in  design  cannot  be  fixed  by  a 
clever  analysis. 

It  is  well-known  tnat  wnile  the  left  brdtri1  plays  a 
conscious  and  dominant  role,  one  may  be  quite  un¬ 
aware  of  the  working  of  the  less  assertive  right  brain. 

For  example,  the  apparently  instinctive  knowledge  of 
what  to  do  and  how  to  do  it  enjoyed  by  an  experi¬ 
enced  tennis  player  comes  from  the  right  brain.  It  is 
significant  that  this  skill  may  be  temporarily  lost  if*  we 
invite  the  tennis  player  to  explain  how  he  does  it,  and 
thus  call  the  left  brain  into  a  dominant  and  interfering 
mode. 

In  this  context  we  sec  the  data  analyst's  insistence 
on  “letting  the  data  speak  to  us“  by  plots  and  displays 
as  an  instinctive  understanding  of  the  need  to  en¬ 
courage  and  to  stimulate  the  pattern  recognition  and 
model  generating  capability  of  the  right  brain.  Also,  it 
expresses  his  concern  that  we  not  allow  our  pushy 
deductive  left  brain  to  take  over  too  quickly  and 
perhaps  forcibly  produce  unwarranted  conclusions 
based  on  an  inadequate  model. 

While  the  accomplishment  of  the  right  brain  in 
finding  patterns  in  data  and  residuals  is  of  enormous 
consequence  to  scientific  discovery,  some  check  is  ob¬ 
viously  needed  on  its  pa  Item -seeking  ability,  lor 
common  experience  shows  that  some  pattern  or  other 
can  be  seen  in  almost  any  set  of  data  or  facts.  A  check 
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VII.  FURTHER  TOPICS 

A.  Graphical  Designs  for  Many  Factors  and/or 
Many  Levels 

Graphical  methods  for  experiment  design  are 
inherently  low-dimensional.  How  can  they  be  used 
to  help  design  experiments  with  many  factors? 

It  may  be  possible  to  develop  low-dimensional 
designs  on  a  subset  of  the  factors  and  'paste'  these 
subset  designs  together  to  form  the  complete  design. 
Pasting  here  means  a  l-l  pairing  between  the  design 
points  for  one  subset  of  factors  and  the  design  points 
for  the  other  subset  The  goodness  of  the  design  will 
depend  not  only  on  the  (graphical)  quality  of  the  two 
subset  designs,  but  on  the  way  in  which  the  subset 
design  points  are  paired.  This  is  illustrated  in  the 
figure  at  the  left.  Two  full-factorial  22  subset 
designs  (a,b  and  c,d)  are  paired  in  two  different 
ways,  yielding  two  different  2<w  designs. 


Some  designs  can  be  projected  onto  a  lower¬ 
dimensional  space.  Draper  (1985)  found  that  a 
Plackett-Burman  (1946)  design  for  1 1  factors  had 
only  two  possible  projections'"  in  5  dimensional 
space.  They  are  illustrated  graphically  at  the  left 


Projecting  designs  into  lower-dimensional  space  can 
help  in  design  analysis,  but  its  value  for  design 
synthesis  remains  unclear. 


'"allowing  sign  changes  and  permutations  of  axes 


b 
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Even  with  relatively  few  dimensions,  say  six,  it  can 
be  difficult  to  visualize  and  develop  a  design  if  the 
factors  each  occur  at  three  or  mote  levels.  In  this 
case  the  nested  cube  approach  described  earlier 
becomes  visually  distracting  rather  than  helpful. 

One  way  to  simplify  the  representation  is  to 
substitute  icons  for  the  smaller  cubes.  The  figures 
here  show  icons  for  small  cubes  for  a  Box-Behnken 
(1960)  design.  Each  small  cube  has  27  possible  sites 
for  an  experiment,  based  on  the  33  plan  for  these 
three  factors. 
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Using  these  icons  gives  this  representation  for  the 
Box-Behnken  design  for  6  factors  at  three  levels. 
Some  characteristics  of  the  design  are  immediately 
apparent  from  its  graphical  representation.  First, 
none  of  the  extreme  comers  of  the  design  space  are 
included  in  the  design.  Second,  the  center  point  is 
not  included. 


Box  •  Bahnkan  3  *  Fractional  Daalgn 
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These  icons  also  make  it  easier  to  synthesize  new 
designs  by  manipulating  the  icon  patterns.  This 
design  has  poorer  properties  than  the  Box-Behnken 
design.  Reasons  become  clear  when  one  looks  at  the 
design  projections.  Those  for  the  front  face  of  the 
large  cube  (projecting  out  d  ■  •.  ..•  .vs)  are  shown  at  the 
left  of  the  figure. 

Again,  these  weaknesses  arc  more  clear  by  looking  at 
the  design  and  its  projections  than  by  trying  to  read 
and  compare  the  tabular  representations  at  the  lower 
right  of  each  figure. 


B.  Mixture  Experiments 

While  mixture  experiments  are  often  associated  with 
chemical  and  petroleum  industries,  they  are  useful  in 
many  resource  allocation  problems  as  well.  They 
can  be  used  for  military  strategy,  medical  treatment, 
and  corporate  strategic  planning. 

Mixture  experiments  involve  factors  whose  sum 
must  be  a  constant.  For  example,  thv  constituents  of 
a  chemical  compound  must  sum  to  100%.  The 
capital  expenditures  for  a  large  corporation  must  sum 
to  the  amount  allocated  for  this  purpose  (if  we 
include  a  dummy  category:  reserves).  Optimal 
weapon  mix  for  strategic  forces  might  be  studied 
through  mixture  experiments,  especially  as  budget 
constraints  become  ever  tighter. 

Design  points  for  a  mixture  experiment  are 
represented  graphically  as  points  on  or  in  a  simplex 
(triangle,  tetrahedron,  etc.),  in  contrast  with  the 
cubes  and  rectangles  used  for  factorial  designs.  The 
figures  at  the  left  show  designs  for  three  and  four 
factors  at  two  and  three  levels  (Cornell,  1981). 

In  addition  to  the  usual  mixture  constraint,  most 
practical  designs  have  additional  requirements  that 
limit  the  design  space.  These  constraints  may  be  to 
focus  the  study  on  reasonable  mixtures,  as  in  the 
figures  from  Koons  and  Wilt  (1985)  and  Hare 
(1985)  at  the  left,  or  they  may  be  due  to  real  limits, 
as  shown  in  the  figure  from  S  nee  (1981)  on  the 
following  page. 
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Figure  4.  Three-Component  Mixture  System  With 
Single  Component  and  Multiple  Component  Con¬ 
straints 
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This  design  is  limited  by  5  bounding  constraints  on 
the  three  components  in  the  mixture,  and  by  two 
multiple-component  constraints.  The  design  points 
selected  for  the  experiment  am  indicated  by  solid 
dots. 

The  computerized  design  software  XVERT  (Snee, 
1981)  uses  geometric  concepts  of  edge?,  vertices, 
and  face  centers  to  select  good  design  points.  This 
approach  to  design  can  be  thought  of  as  mental 
graphics ,  since  a  graphical  image  is  used  but  is  not 
actually  drawn. 


C.  Incomplete  Block  Designs 

A  graphical  representation  for  an  incomplete  block 
design  was  presented  on  page  27.  A  blocking 
variable  is  thought  of  much  as  any  other  factor, 
except  that  a)  it  is  qualitative,  not  quantitative,  and  b) 
there  are  usually  more  than  two  values  (blocks). 

The  blocking  factor  is  usually  qualitative,  not  ^ 
quantitative.  This  makes  checking  for  confounding 
patterns  more  difficult  The  graphical 
subcomponents  corresponding  to  each  block  must  be 
shuffled  around,  mentally  at  least,  to  identify 
confounding  patterns.  The  confounding  on  page  27 
is  seen  by  pairing  the  second  and  third  blocks  against 
the  1-4  pair. 

'Mental  graphics'  yields  several  general  classes  of 
partially  balanced  incomplete  block  designs.  These 
are  described  at  the  left  by  Kempthome  (1952). 


D.  Network  Representations 

The  edges  of  a  multidimensional  point  plot,  along 
with  the  design  points,  make  up  a  network  that 
directly  represents  the  design.  There  are  also 
indirect  network  representations  that  can  be  useful  in 
creating  and  analyzing  designs. 
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blooks  treatment 

levels 


This  network  represents  the  allocation  of  factor  levels 
to  blocks.  Butz  (1982)  relates  connectivity  in  such 
networks  to  estimable  contrasts  of  factor  levels. 
These  plots  can  be  used  to  develop  and  analyze 
designs  for  use  with  ANOVA  models. 


ALL  TREATMENT 
CONTRASTS 
ESTIMABLE 


BIPARTITE 
GRAPH  IS 
CONNECTED 


As  mentioned  on  page  53,  Taguchi's  'linear  graphs' 
provide  analytical  insight,  but  do  not  really  help 
generate  designs.  They  illustrate  confounding 
patterns  graphically  by  labeling  nodes  and  edges  with 
effects.  For  example,  the  triangle  at  the  upper  left 
indicates  that  the  main  effect  of factor  5  will  be 
confounded  with  the  1-4  interaction  effect  in  that 
design. 

The  linear  graphs  make  cookbook  design  selection  a 
more  effective  approach.  A  catalog  of  these  network 
representations  makes  it  possible  to  select  a  design 
with  confounding  patterns  that  match  the  particular 
needs  at  hand. 

Sources:  Taguchi  (1980)  and  Pignadello  and 
Ramberg  (1985) 


..?•  u, +ABc  *««« ^ 
>i  l+ABC.  \\  riitin  inii  la  man  dl„||,  ,h,„  nntun, 


■T  +  « 
*  +  .4C 
C  +  AS, 


KlpMllitly. 

tJI  !tiiri!uJL‘uhli,.i!lll,*f  ri,T  >k*  *i«  •»*«  im  iipw. 
2!? if?"*! 1,11  ^  ">•  f*1"  “»>  *»  <»»!«»•«. 

nut  to, «  ho«  nia  la  ihuuiM  by  ih.  »m,pendla,  Tbl.  ,r,Mla* 


Kli.  I.  CaaUuidiB'  pitumt  if  It,  Uf  3**>  1,4  i*-«. 


Cuthbert  Daniel  (1962)  also  used  networks  to 
represent  confounding  patterns  in  fractional  factorial 
designs.  These  are  harder  to  decipher.  Their 
usefulness  seems  to  be  for  design  analysis,  not 
design  synthesis. 
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Graphical  representations  are  helpful  in  developing 
nested  designs  for  random  and  mixed  effects  models. 
The  simple  schematic  at  right  makes  the  sampling 
pattern  clear.  Leone,  et.  al.  (1968)  use  these  simple 
figures  to  present  their  hierarchical  designs. 
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Andrews  used  more  elaborate  representations  for 
nested  designs.  The  figures  take  more  time  to  draw, 
but  they  can  prompt  the  experimenter  to  think  about 
important  procedural  or  design  issues.  This 
elaborate  figures  may  well  have  merit  over  its  simpler 
counterpan  above. 


E.  Nomograms 

Nomograms  are  graphical  aids  for  computation.  In 
experiment  design  tney  can  be  used  for  design 
(choosing  sample  sizes,  operating  conditions)  or  for 
analysis  (identifying  design  properties  such  as 
variance  of  the  estimated  parameters).  In  the 
example  at  the  left,  Villars  (1951)  shows  the  relative 
efficiency  of  ANOVA  designs  as  a  function  of  the 
number  of  levels  of  the  treatment  factor  and  the 
number  of  replications. 
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This  nomogram  from  Nelson  and  Kielpinski  (1975) 
is  for  high  temperature  accelerated  lifetests.  It  allows 
one  to  identify  the  variance  of  a  predicted  response  as 
a  function  of  test  temperatures  and  sample 
proportions. 


Nomograms  can  be  modified  graphically  to  provide 
alternative  or  additional  information.  This  allows 
one  to  optimize  other  properties  (than  prediction 
variance)  by  adding  graphical  constructs  to  the 
figure.  This  example  allows  the  experimenter  to 
choose  test  conditions  that  will  yield  the  required 
accuracy  while  minimizing  the  maximum  test 
temperature.  This  is  important  because  high  test 
temperatures  can  introduce  failure  mechanisms  that 
are  not  active  in  the  normal  operating  range. 


Source:  Barton  (1987) 
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There  are  many  nomograms  aid  in  sample  size 
determination.  The  example  at  the  left  ts  from  Beech 
(1961)  for  the  selection  of  sample  size  to  estimate  the 
regression  coefficient  to  a  desired  precision. 
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